Skip to main content
General

AI Detectors: What a "100% AI-Generated" Score Is Worth

A detector declared a prize-winning short story "100% AI-generated." On essays by non-native English students, these tools wrongly accused more than half of the human texts. What this case says about how writers get accused today.

5 months ago8 min read

On 12 May 2026, the British literary magazine Granta, founded in 1889 and the first home of many of today's great English-language writers, published the five regional finalists of the Commonwealth Short Story Prize. The Commonwealth Foundation awards the prize every year to unpublished stories from the fifty-six Commonwealth countries, and a few days after publication, one of those stories set off a controversy.

"The Serpent in the Grove," by Trinidad's Jamir Nazir, the Caribbean regional winner, follows a rum-loving farmer who stumbles onto an enchanted grove. Several readers found the prose too smooth, and Ethan Mollick, a Wharton professor and widely followed commentator on AI, ran it through Pangram, an American tool that claims to spot machine-written text. Pangram answered: "100% AI-generated."

A number that round looks like proof, which is exactly the problem, because a detector never produces proof.

What an AI detector calculates

A detector measures a text's "perplexity," meaning how predictable each word was given the words before it. It follows neither the story nor the author's intent, and the more closely a sentence sticks to the most probable path, the more likely the tool is to decide a machine wrote it.

Someone writing in an English learned at school, without a native speaker's irregular turns of phrase, inevitably writes more predictable sentences. A Caribbean story written in careful, slightly formal English therefore ticks the boxes a detector associates with a machine, even though its author was simply being careful.

A study measured this. In 2023, a Stanford team ran TOEFL essays, the English test taken by non-native students, through seven widely used AI detectors, and 61.22% of these human-written essays were classified as "AI-generated." Each of the seven detectors wrongly accused some of the students, which amounts to saying that these tools mostly flag people who write cleanly in a language that is not their own.

Pangram came out after that study and isn't in it, and its makers report much lower error rates. The principle stays the same: the tool calculates a probability, and displaying it as "100%" makes it look like a verdict. Between a probability and a proof lies the same distance as between a suspicion and a conviction.

Granta asks an AI whether the text came from an AI

Pressed to explain, Sigrid Rausing, Granta's publisher, said her team had shown the story to Claude, Anthropic's assistant, and asked whether an AI had written it. Claude replied that the text was "almost certainly not produced unaided by a human."

A major literary magazine asked an AI to say whether a text came from an AI, so one algorithm brought the accusation and a second one handled the defense, and no human reader settled the question at any point.

The Commonwealth Foundation eventually took the only position it could hold. Every finalist had declared on their honor that they wrote their story alone, and since no reliable tool can check that, the foundation explained that it "must operate on the principle of trust." In other words, it doesn't know, and nobody can.

Three situations the debate lumps together

The debate quickly hardened into two camps. Opponents of AI say detectors have unmasked cheats and that machine-written texts should be kept out of prizes; supporters answer that detectors are pseudoscience and that the question hardly matters. Both camps argue over the same question, "did AI write this text, yes or no?", when it actually covers three very different situations.

In the first, the writer uses AI as a tool: asking for a synonym, having it reread a paragraph, trying out a phrasing, and rejecting most of what it suggests. Developers have worked this way for two years, and nobody claims they stopped writing their own code, since they decide on every line that goes into it. A novelist who works like this remains the author of the book for the same reason.

The second situation is the one that raises a real problem, and it now has a name, vibe writing, by analogy with the "vibe coding" of developers who send prompts to the machine and ship whatever comes out without reading it. A vibe writer generates two hundred and eighty pages from a prompt and puts their name on the cover, which raises a question of honesty toward the reader as much as one of authorship.

The third situation is the newest and the least discussed: a writer who had no help at all gets accused because their prose, too clean or written in a learned language, tripped a detector's alarm. That is the person being convicted on the strength of a number.

What the writer is accused of, and what they can't prove

Opponents of AI need reminding that a detector score proves nothing. Graphology and the lie detector took decades to leave the courtroom, and a percentage printed at the end of a calculation doesn't make the method any more rigorous. It is all the more serious because this measure penalizes, first of all, people writing in a learned language, who are very often writers from former colonies, precisely the writers this prize exists to bring to light.

Supporters of AI need reminding that a prize reserved for unpublished human fiction means something, and that writers' distrust is legitimate even when the tool meant to address it works badly. You can hold both ideas at once: the detector is bad, and the question it claims to settle is real.

The most worrying part of this case is that a writer can now be accused on the strength of a number nobody can challenge. The burden of proof has flipped without anyone deciding it should, and it now falls to the writer to show they didn't cheat. Yet nobody can prove they didn't use AI, any more than anyone can prove an absence in general.

The detector, for its part, can't tell the first situation from the second, since it can give the same score to the writer who asked for a synonym and to the one who generated the whole book. It only sees how predictable the sentences are, while the line it's asked to police runs through the writer's work.

Showing how the text was written

If detection doesn't work, one solid approach remains, which is to look at where the text came from rather than at how it looks. Instead of asking whether a text seems human, a question no machine will ever answer honestly, you ask whether it's possible to show how it was written: the history, the successive versions, the deleted passages, the draft turning into a page. It's less spectacular than a red percentage, and anyone can check it.

That's why several online writing apps, including WriteControl, Extypis and Scribbook, now offer an AI-free mode. For a student facing an integrity check, or a writer who wants to be able to show that a text is entirely their own, it's the only answer that holds up against an unreliable detector, because it lets them show the path they took rather than defend themselves after the fact. A piece of software offering AI says nothing, in itself, about how the writer uses it.

Jamir Nazir swore he used no AI. He may well be telling the truth, and we will probably never know, which should have been enough to keep a respected literary foundation from putting the question to a robot. As long as a probability passes for proof, innocent writers will keep being accused in the hope of catching a few cheats.


Sources: Granta ("The Serpent in the Grove," May 2026); Literary Hub; Futurism; Boing Boing; Scroll.in; Slashdot; Caribbean360; Liang et al., "GPT detectors are biased against non-native English writers," Patterns (Cell), 2023; Stanford HAI; Commonwealth Foundation.

HU

Hubert Giorgi

Author

Write with Extypis

The writing studio novelists were missing

Outline, characters, places and objects, the links between them, mind map, scene board, export: everything lives in one place. Plus an AI that knows your story, whenever you want one.

Discover Extypis

Free to write and to export. No credit card asked.