AI in Education
Module 7. Can AI Be Detected?
The quick answer is no, chatbots cannot reliably be detected. Here the keyword is reliably. Some companies claim to have developed chatbot detectors but when used in the real world, chatbot detectors err on both sides of the equation by saying that AI written articles are original, or conversely, claiming that original articles were written by AI. Here is an article from ZDNET in which seven out of nine ChatGPT detectors failed to work reliably:
In the following Medium article, writing teacher Whitney Gegg-Harrison has a similar experience. After having this very article judged as having been written by AI, Gegg-Harrison concludes that chatbot detectors are unhelpful and counter-productive. Along the way she questions equity issues related to the algorithms that AI Chatbot detectors use:
The artificial intelligence company that created ChatGPT is called OpenAI. In order to stop kids from cheating, OpenAI created an AI Text Classifier designed to detect whether an essay got written by ChatGPT:
Plagiarism detection company TurnItIn likewise entered the fray, claiming to have invented a new technology that can detect student use of AI:
Not everyone opposes student use of AI. The following article is by a writing professor who believes that students should never be required to have their essays scanned by AI writing detectors. Ironically, when this professor had an AI writing detector scan this article, the AI said the professor’s article was likely to be written entirely by AI. Follow this link and see what you think:
In the following open access research article, a group of University of Maryland computer science professors and doctoral candidates explain why AI generated text cannot be reliably detected:
- Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., & Feizi, S. (2023). Can AI-Generated Text be Reliably Detected? https://arxiv.org/abs/2303.11156
AI Detection Tools
Listed here are tools that claim to be able to detect student use of AI:
The following articles discuss the validity and impact of AI detection tools:
- Reinstein, Julia. (2024). Parents sue school in Massachusetts after son punished for using AI on paper. ABC News.https://abcnews.go.com/US/parents-sue-school-massachusetts-after-son-punished-ai/story?id=114819025
- Munoz, Roberta. (2024). The Problem with “Perfect” Answers: GenAI and Academic Research Tools. Educause Review. https://er.educause.edu/articles/2024/10/the-problem-with-perfect-answers-genai-and-academic-research-tools
- Reporting that 20% of Black teens are falsely accused of cheating on an assignment with AI compared with 10% of Latino teens and 7% of white students. Common Sense Media. (2024). The Dawn of the AI Era: Teens, Parents, and the Adoption of Generative AI at Home and School. https://www.commonsensemedia.org/sites/default/files/research/report/2024-the-dawn-of-the-ai-era_final-release-for-web.pdf
- Seetharaman, Deepa & Barnum, Matt. (2024). There’s a Tool to Catch Students Cheating With ChatGPT. OpenAI Hasn’t Released It. Wall Street Journal. https://www.wsj.com/tech/ai/openai-tool-chatgpt-cheating-writing-135b755a
- Ofgang, Eric. (2024). 6 Ways Teachers Can Tell Students Are Using AI. Tech & Learning. https://www.techlearning.com/how-to/6-ways-teachers-can-tell-students-are-using-ai
- Gewirtz, David. (2024). I tested 9 AI content detectors - and these 2 correctly identified AI text every time. ZDNet. https://www.zdnet.com/article/i-tested-7-ai-content-detectors-theyre-getting-dramatically-better-at-identifying-plagiarism/
