In July 2023, the journal Patterns published a short paper by Weixin Liang, James Zou and colleagues at Stanford, with a blunt title: “GPT detectors are biased against non-native English writers” [1]. The team ran human-written essays through seven widely used detectors at a time when educators were worried about spotting AI-written work [1]. Essays by US eighth graders were classified correctly. More than half of the essays written by non-native English speakers for an English proficiency test were labeled as AI-generated [1].
My view is simple: a detector score is a rough, probability-like signal about how predictable a piece of text looks. It is not evidence of cheating, and treating it as a verdict puts the heaviest burden on the students and researchers who are already writing in their second language.
What the Stanford team tested
The researchers took 91 essays written for the TOEFL, the Test of English as a Foreign Language, collected from a Chinese forum, and 88 essays by US eighth graders from a public dataset [1]. All of them were written by people. Then they ran every essay through seven detectors [1].
The earlier preprint version of the study names them: Originality.AI, Quil.org, Sapling, an OpenAI detector hosted on Hugging Face, Crossplag, GPTZero and ZeroGPT [2]. That OpenAI entry is not the separate classifier OpenAI launched in January 2023, which I come back to below. That preprint went online in April 2023, so these results describe those tools as they were in early 2023 [2]. Detectors get updated all the time, and I am not claiming that every tool on the market today behaves the same way.
The detectors did well on the American students’ essays. On the TOEFL essays, the average false positive rate was 61.3% [1]. All seven detectors agreed that 19.8% of the human-written TOEFL essays were AI-written, and at least one detector flagged 97.8% of them [1].
That last number stays with me. A school that acted whenever any one of these seven tools raised a flag would have accused almost every one of these writers.
Why careful, simple English looks like a machine
Most detectors lean on a measure called perplexity. In plain terms, it describes how surprised a language model is when it tries to guess the next word [1]. AI text tends to be predictable, so low perplexity is read as a sign of a machine. The trouble is that writers with a smaller working vocabulary also tend to use common words in common orders, and the flagged TOEFL essays had significantly lower perplexity [1].
The team then tested this directly. When they used ChatGPT to enrich the vocabulary of the TOEFL essays, the average false positive rate fell from 61.3% to 11.6% [1]. When they did the reverse and simplified the eighth graders’ word choices, misclassification went up sharply [1]. So word choice alone moved the detectors a lot, whoever or whatever actually wrote the text.
Imagine an international graduate student who writes her thesis chapter slowly, in plain sentences, because she wants every claim to be clear. The software flags it as likely AI. A classmate pastes in a chatbot draft, asks it to sound more literary, and sails through. The tool has rewarded the shortcut and punished the honest effort.
That scenario is not far from what the study measured. The authors asked ChatGPT to write college application essays, and at first the detectors caught them. Then they asked it to revise its own text with the prompt “Elevate the provided text by employing literary language,” and detection rates fell to near zero [1].
The tool makers said it too
OpenAI released its own classifier in January 2023 and said up front, “Our classifier is not fully reliable.” On OpenAI’s own challenge set of English texts, it correctly flagged only 26% of AI-written text and labeled human writing as AI 9% of the time. It also said the tool “should not be used as a primary decision-making tool.” In July 2023 OpenAI added a note to the same page: “As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy.”
Turnitin published its own numbers in a May 2023 update. By mid-May it had run 38.5 million submissions through its AI detector. It said real-world use was giving different results from its lab, and that it found a higher rate of false positives when it detected less than 20% AI writing in a document. For documents above that level, it put its document false positive rate at less than 1%, which it said means it “might flag a human-written document as having AI writing for one out of every 100 human-written documents.” It added that “behind each false positive instance is a real student who may have put real effort into their original work.”
One in a hundred sounds small until you remember how many essays a large university grades in a single term. And Turnitin’s figure came from its own testing, not from a comparison like Liang’s between native and non-native writers.
The case for keeping detectors
I understand why teachers want these tools. AI writing is easy to produce, and a number on a screen feels objective. The Stanford authors do not say detectors are useless either. They suggest one lower-risk use: as a self-check that helps students spot overused phrases and clichés in their own drafts [1].
I agree with that. A detector can be one input among several. What it cannot be is the deciding input, because it measures a property of the text, not an act by a person. The authors point out that a false accusation of cheating can harm a student’s career and well-being, and that the damage can stick even if the accusation is later withdrawn [1]. They also warn that journals and conferences that ban AI writing could end up penalizing researchers from countries where English is not the first language [1].
Part of my own work is building large language model tools that extract structured information from medical records. Their outputs are predictions that can be wrong, so the useful question is always how that output gets checked before anyone acts on it. A detector’s output deserves the same treatment.
What a detector score should trigger
If you teach, review or edit, treat a high AI score the way a doctor treats an abnormal screening result: as a reason to look more closely, not as a diagnosis. Ask for drafts, notes or version history. Talk with the student about the argument. Be especially careful when the writer is working in a second language, because that is exactly where the tools in the 2023 study failed most often [1].
A detector score is a guess about how predictable some text looks, not proof of what a person did. Using it as a verdict punishes careful non-native writers while clever cheaters walk past, and that is the opposite of integrity.
If you are a student or researcher who has been flagged, it is fair to ask what tool was used, what its known error rates are, and what other evidence exists. Academic integrity is a judgment about people, and people deserve more than a percentage. The 2023 tests showed us why, and even OpenAI and Turnitin warned about the errors in their own tools.
References
[1] W. Liang, M. Yuksekgonul, Y. Mao, E. Wu, and J. Zou, “GPT detectors are biased against non-native English writers,” Patterns, vol. 4, no. 7, Art. no. 100779, Jul. 2023, doi: 10.1016/j.patter.2023.100779.
[2] W. Liang, M. Yuksekgonul, Y. Mao, E. Wu, and J. Zou, “GPT detectors are biased against non-native English writers,” arXiv:2304.02819, Apr. 2023, doi: 10.48550/arXiv.2304.02819.
Related Articles
