AI Detector Arms Race: Why No Single Score Is Reliable

The AI detector arms race is a loop: generators change how text is produced, detectors retrain on that output, generators adapt again. No single score survives that loop, because each score describes a moving target. The practical answer is to treat detector output as one weak signal among several, read it at sentence level, and decide what to do next rather than treating a number as a verdict.

Why do detectors and generators keep leapfrogging?

Both sides are optimising against the other’s last move.

A generator’s goal is fluent, useful text. A detector’s goal is to separate that text from human writing. When a new model ships, detectors lose ground until they collect enough of its output to retrain on. When detectors retrain, the next model release tends to sand down the habits that were easiest to catch: the repeated sentence shapes, the over-even rhythm, the predictable connector words.

This is not sabotage on either side. It is what happens when two products share one measurement. Detectors are trained on samples of model output; models are tuned on human feedback that rewards smooth, generic prose. Smooth, generic prose is exactly what a detector learns to recognise. So the generator’s improvements and the detector’s signals keep chasing each other around the same circle.

There is a second reason the loop never closes. Detection is statistical. A detector is not reading your intent, your sources, or your process. It is comparing your text to patterns it has seen before and returning a probability. That probability shifts when the training data shifts, even if your writing has not changed at all. A draft that scored one way last term can score differently this term without a single word being edited.

Our own instrument shows how noisy this gets. On 2026-09-28 we scored two corpora with the DeAIze detector (local ensemble, lite tier) at a 30% flag threshold. A human-written sample of 709 hand-written DeAIze guides had a mean AI score of 26% and a median of 25%, with a range of 0-69%; 239 of those 709 were flagged at that threshold. Unedited model output, 12 passages generated from fixed prompts and not edited, had a mean of 33% and a median of 34%, range 4-46%.

Read that again, because it is the whole article in one paragraph. The two distributions overlap heavily. A large share of genuinely human writing sits inside the range where unedited model output also lives. That is not a broken detector. It is what a probability estimate over short, formal, edited prose looks like.

What does that mean if you are relying on one score?

It means a single number cannot carry the decision you are asking it to carry.

If you are a student, a teacher, an editor, a hiring manager, or a freelancer, the tempting move is to run one tool, read one percentage, and act. That move fails in both directions.

False positives happen when human writing lands high. Formal register, heavy editing, translation, non-native phrasing, and short samples all push scores up. A grant proposal written in tight institutional English can look statistically similar to model output, because both are low-variance, high-consistency prose.

False negatives happen when model output lands low. A passage that has been lightly rewritten, or that simply drew a low probability from this particular model on this particular day, can pass. The absence of a flag is not evidence of authorship any more than the presence of one is.

Both errors are the same error seen from opposite sides: you are asking a probability to do the job of a fact.

If you want to see how sentence-level scoring changes the picture, our guide to how AI detection works walks through the signals, and the AI detector hub explains what each part of a report actually measures. Neither one asks you to trust a single number.

Why do two detectors disagree about the same draft?

Because they are not measuring the same thing, even when they use the same words.

What differs	Why it changes the score	What you can do
Training data	Each tool learned from different samples of model output and human writing	Run more than one tool before drawing a conclusion
Granularity	A document score hides which lines carried the signal; sentence scores show them	Prefer tools that highlight per sentence
Threshold	A flag at a low threshold and a flag at a high threshold mean different things	Ask what threshold produced the flag
Sample length	Short passages give a detector less to work with	Judge short text cautiously
Language and register	Non-English text and translated text behave differently	Check a tool’s language coverage before trusting it

This is why the reliability question is the wrong question in its usual form. “Is this detector accurate?” has no stable answer, because accuracy is measured against a fixed test set and the generators keep moving. The better question is: what decision am I making, and how much weight can this signal bear?

For a low-stakes decision, like deciding which paragraphs of your own draft to rewrite, a detector score is useful. For a high-stakes decision, like accusing someone of misconduct or rejecting a submission, a detector score alone is not enough. It was never designed to be.

Where does the honest use of these tools settle?

It settles on three habits.

1. Score at sentence level, not document level. A whole-document percentage tells you something moved. Sentence highlighting tells you where. That is the difference between a verdict and an edit list, and only one of those is actionable.
2. Treat the score as a prompt, not a proof. A high score means “look closer here”. It does not mean “this was generated”. A low score means “nothing stood out to this model today”. It does not mean “this was written by a person”.
3. Keep the process, not just the file. Drafts, notes, sources, version history, and a short statement of how you used any AI assistance do more to establish authorship than any detector output. If you are ever asked to account for a piece of writing, the paper trail is the evidence.

This is also where the honest boundary sits. Reducing false positives on your own writing is fair. Cleaning up your own AI-assisted draft so it reads like you is fair. Passing off someone else’s work, or using a humanizer to disguise authorship you should be disclosing, is not, and no tool changes that. The ideas, data, and conclusions have to be yours.

How should you interpret a detector result responsibly?

Use a short checklist every time you see a score.

• Ask what was measured. A document score and a sentence score are different products. Know which one you are looking at.
• Ask what threshold applies. A flag at a low threshold is a weaker signal than a flag at a high one.
• Look at the flagged sentences, not the headline number. Read them aloud. Do they sound like you, or do they sound generic? That judgement is often more informative than the score.
• Check for the usual false-positive triggers. Formal register, translation, heavy editing, and short samples all raise scores on human writing. Our own human sample of 709 hand-written guides averaged 26%, which is a long way from zero.
• Run a second tool when the stakes are high. Disagreement between tools is normal and it is information: it tells you the signal is weak.
• Never present a score as proof of authorship. It is a probability estimate. Say so when you pass it on.

A worked illustration, using a made-up example rather than a real figure: imagine a paragraph of your own institutional prose scores high on one tool. You open the sentence view and see that the two flagged lines are the definition sentence and the closing summary, both written in the same even, low-variance register you use everywhere. You rewrite those two lines to match how you actually speak, re-score, and the flagged set shrinks. Nothing was hidden. You just removed the ambiguity that made your writing look like everyone else’s.

What does the future of AI detection look like?

The arms race does not end. It becomes less central.

Three shifts are already visible. Detection is moving toward provenance and disclosure rather than pure classification, because a declared process is more reliable than a guessed one. Tools are moving toward sentence-level and editing-oriented output, because that is what people can actually act on. And institutions are slowly moving toward policies that describe what is allowed, rather than policies that depend on a score holding still.

For anyone relying on a single score, the takeaway is unglamorous. Use detectors as one signal. Read them at sentence level. Keep your drafts. Disclose what you used. And when a score disagrees with what you know about your own writing, trust the evidence you can actually show, not the number.

---

Measurement note: figures in this article come from our own detector run on 2026-09-28 — n = 709 hand-written guides versus n = 12 unedited model outputs, median AI score 25% and 34% respectively. Method: /methodology/.