I keep seeing people share screenshots of AI detectors with a kind of certainty that does not quite match how these tools actually work. A score appears on the screen, percent human, percent AI, and suddenly that number becomes a verdict.
I find myself pausing whenever I see that, wondering how we reached a point where a statistical guess is treated as a form of truth.
It feels as if these detectors were given more authority than they were ever designed to carry.
The assumption most people make is simple.
If a detector says something is AI generated, then it must be.
But the more I look at the research, especially the work that has come out in 2024 and 2025, the more obvious it becomes that these tools are not detecting authorship.
They are measuring patterns.
And patterns do not tell you who wrote something.
They barely tell you how it was written.
AI detectors mostly rely on two metrics, perplexity and burstiness.
Perplexity is a measure of how predictable a piece of writing is. Burstiness looks at variation between sentences.
Clean, consistent writing often looks artificially predictable to these tools, even when it is written by a human. And rough, inconsistent writing often looks human, even when it comes from an AI system that has been lightly edited or paraphrased. It is not surprising that this approach leads to strange outcomes.
In June 2025, the National Centre for AI reviewed several widely used detectors and found that none delivered reliable accuracy across different writing styles or contexts. The report noted that published studies often use small sample sizes, and actual performance in real world conditions is far lower than marketing claims suggest. (Source)
Another review from early 2025 aggregated results from thirty four different papers. The authors concluded that while many detectors perform above chance, none consistently identify AI text when the writing has been edited, paraphrased or mixed with human contributions. They also noted persistent bias against non native English writers. (Source)
What caught my attention even more was a 2024 study testing detectors on hybrid writing, meaning text created partly by AI and partly by a human. The failure rate was so high that detection essentially collapsed. Once a person makes small edits, the statistical patterns detectors rely on begin to blur. The writing becomes something else, something that does not fit neatly into either category. (Source).
Educators have been vocal about this problem too. The University of Minnesota published guidance in 2025 advising faculty not to use detectors as the primary method for judging student work. They highlighted issues with false positives, bias, and the inability of detectors to account for natural variation in human writing. (Source)
Legal and academic librarians have raised similar concerns. A 2025 summary from the University of San Diego noted that detectors often present themselves with unwarranted confidence, yet produce both false positives and false negatives at rates too high for responsible use in disciplinary or professional settings. (Source)
It is not just researchers who see the problem.
Many writers have discovered it by accident.
I remember reading about an instructor who fed classic literature into a detector out of curiosity.
The results came back as mostly AI generated. A separate case involved a student punished for supposedly submitting AI written work, even though the writing style matched their previous assignments.
The common thread in these stories is not dishonesty. It is the mismatch between what detectors claim to measure and what they actually measure.
And maybe that is the part that keeps me thinking the longest. Writing is fluid. It is shaped by experience, editing, personality, context, fatigue, inspiration.
An algorithm trained on patterns cannot understand any of that.
It cannot tell whether someone wrote while stressed or tired or in a hurry.
It cannot know whether an AI assistant drafted one paragraph while the human wrote the rest.
It only sees predictability, variation, and patterns that sometimes resemble one category and sometimes resemble another.
That is simply not enough information to make definitive claims.
There is also a deeper ethical problem.
When detectors produce false positives, the consequences fall on real people.
Someone applying for a job might be misjudged.
A student might be accused of misconduct.
A writer might be told their voice sounds too artificial.
These are not harmless mistakes.
They shape trust, opportunity and reputation.
What feels more honest is to recognise what detectors can and cannot do.
They can flag text that looks highly formulaic or machine produced, at least in straightforward cases.
But they cannot reliably tell the difference between a thoughtful human writer, an AI system producing polished text or the increasingly common reality of hybrid writing.
They cannot see the human intention behind the words.
So when people ask whether AI detectors work, I think the fairest answer is that they work in narrow, controlled situations, but not in the broader world where writing is messy and collaborative and constantly revised.
And not in the sense of revealing authorship, which is what many people assume.
If anything, the research from 2024 and 2025 seems to say the same thing.
These tools are interesting as experimental technology, but they are not mature enough to be used as gatekeepers.
They are not reliable enough for policing.
And they are not subtle enough to understand the difference between clarity and artificiality.
It feels healthier, both for readers and for writers, to return to a simpler test.
Does the content feel thoughtful. Does it help someone. Does it communicate something clearly. That is where the value lives.
Not in whether a detector believes the writing is human, but in whether the reader does.
Share this post