Is AI Detection Evidence? What Due Process Requires

No. Is AI detection evidence of authorship? It is not. A detector score is a probability estimate produced by a statistical model — one weak signal among several. Treat it as a reason to look closer, never as a finding. The moment a score is used to accuse someone, you owe them process: disclosure, a chance to respond, and a documented decision.

Key takeaways

• A detector score is a probability, not a verdict. It cannot tell you who wrote a sentence, only how much the text resembles model output.
• In our own measurement (2026-09-19, n = 46 hand-written DeAIze guides, 30% flag threshold), human-written text averaged an AI score of 27%, with 17 of 46 flagged. That is the false-positive problem in one line.
• Due process means the accused sees the score, knows which tool produced it, and gets to answer before any penalty.
• Document the reasoning, not just the result. “The score was high” is not a decision record.
• Ask what else corroborates the flag: drafts, version history, oral defence, citation checks, writing history.

Why is AI detection evidence so weak on its own?

Detectors look at surface statistics: how predictable each word is, how much sentence length varies, how often the text repeats a pattern. Those signals correlate with machine generation. They also correlate with careful academic prose, translated text, formulaic legal writing, and the plain English that non-native writers are often taught to use. That overlap is why a score can be high on a document no model touched.

Our local ensemble, run on 46 hand-written DeAIze guides at a 30% flag threshold, produced a mean AI score of 27% and a median of 25%, with a range from 0 to 55. Seventeen of those 46 human documents were flagged. Nothing was generated. The scores simply landed above the line.

The same run scored 12 unedited model passages: mean 35%, median 34%, range 25 to 49. Notice the overlap. Human writing and unedited model output occupy neighbouring bands. A single number cannot separate them cleanly, and any process that pretends otherwise will punish the wrong people.

One more figure from that run is worth sitting with. AI-tell frequency — the density of stock phrases, hedges and filler — was 3 per 100 words in the human sample and 1 per 100 words in the unedited model output. The humans used more tells than the machines. If your instinct is “I can just tell by reading,” that instinct is measurably unreliable too.

What does due process look like when a score is used against someone?

Due process in this context is not a legal formula. It is a sequence that gives the accused a fair chance to answer. For a teacher, editor or HR writer, it looks like this:

1. Separate the flag from the finding. Record that a tool returned a score. Do not record that the person used AI.
2. Disclose the tool and the score. Name the detector, the version if known, and the exact reading. Secrecy about the instrument makes appeal impossible.
3. Show the sentence-level breakdown. A whole-document number hides which lines carried the signal. If your tool only gives a document score, say so — that limitation belongs in the record.
4. Invite a response before deciding. Ask the person to explain their process, show drafts, or discuss the content orally.
5. Weigh corroborating evidence. Version history, earlier work in the same voice, cited sources, and the person’s ability to discuss the material all matter more than the score.
6. Decide and write it down. State what you relied on, what you discounted, and why.

If you are the person on the receiving end, the practical mirror of this is covered in False Positive AI Detector: What to Do When You’re Accused. The steps are the same; only the seat changes.

If you are the one running the check, run it on your own process first. DeAIze’s detector shows a document score and highlights each sentence by AI probability, so you can see whether a flag sits on one odd paragraph or across the whole piece — which is exactly the detail a fair decision needs.

Which questions should you ask before acting on a flag?

Work through these before any consequence attaches. If you cannot answer them, you are not ready to act.

Question	Why it matters
Which tool, which version, which threshold?	Scores shift between tools and settings. A flag at one threshold may not exist at another.
Is the score on the whole document or on specific sentences?	A single flagged paragraph is a very different signal from a uniformly flagged essay.
Does the person have a documented writing history?	Prior work in the same voice is strong counter-evidence.
Could translation, template language, or plain-English instruction explain it?	These are known sources of false positives.
What happens if the score is wrong?	If the answer is “irreversible harm,” your evidence bar needs to be higher.
Has the person been told the score and given a chance to reply?	If not, stop. This is the process.

That last row is not a formality. A decision made before the person has seen the allegation is not due process, whatever the paperwork says.

How do you document a decision that relies partly on a detector?

Write the record as if someone will read it in two years with no memory of the case. A usable record has four parts:

• What the tool reported. Tool name, date, score, threshold, and whether the reading was document-level or sentence-level. Note that a detector score is a probability estimate, not proof of authorship.
• What else you looked at. Drafts, revision history, oral discussion, source checks, prior work. List what you found, including anything that cut against the flag.
• What you concluded and why. Distinguish between “the score was high” and “the score was high, the person could not explain the argument, and no drafts existed.” Only the second is a finding.
• What you did. The action, its scope, and the appeal route.

Keep the raw score out of any public or permanent file unless the policy explicitly requires it. A probability estimate that later proves unreliable should not follow someone for years.

Editors and content teams can borrow the same discipline. If a freelance draft gets flagged, the record is the brief, the revision history and the writer’s own notes — not the number. Our guide to how AI detection works explains which signals feed the score, which helps you judge how much weight it deserves in a given genre.

What do you tell a student or employee who disputes the score?

Start by conceding the real limitation. Detectors are probabilistic. They disagree with each other. They flag human writing. Say that out loud, because it is true, and because pretending otherwise destroys trust the moment they search the topic themselves.

Then do three things:

1. Show them the report. The score, the sentences that carried the signal, and the tool’s own stated caveats.
2. Explain what would change your mind. A draft trail, a short oral walkthrough of the argument, earlier work in the same register. Give them a route, not a wall.
3. Separate the process from the outcome. You can believe a flag was worth investigating and still conclude the person wrote the work. Saying so is not weakness; it is the system working.

If the dispute continues, point them at the appeal route in writing and keep your own notes factual. For academic cases, Academic Integrity Hearing AI: What Panels Assess walks through what panels actually weigh — usually explanation quality and corroboration, not the raw number.

Why do teachers and panels still get this wrong?

Because a number feels objective. It is produced by software, it has a decimal point, and it arrives before any human has read the work. That is exactly backwards. The score should be the last thing you look at, after the draft trail, the conversation and the content itself.

There is also asymmetry in the harm. A false positive costs the accused enormous time and stress; a false negative costs an institution very little. Systems optimised for catching everything will generate false positives by design. If your policy does not name that trade-off, it is not a policy — it is a preference.

Policy writers should also check whether their rules even permit what they are doing. Academic Integrity Policy AI: What Your University Actually Allows is a useful sanity check on that, and AI Detection vs Plagiarism: Two Different Failures clarifies why a plagiarism finding and an AI flag are not the same accusation.

What should your written policy actually say?

A defensible policy names the tool, the threshold, the human review step and the appeal route. It states that a detector score alone will not trigger a penalty. It says who may see the score and how long it is retained. It requires the accused to be shown the evidence before a decision.

It should also say what the institution will not do: it will not treat a probability estimate as proof, and it will not act on a tool the accused is not allowed to see. Those two sentences prevent most of the cases that end up in the news.

Finally, build in a review date. Detector behaviour changes, thresholds get retuned, and a policy written around one tool’s output in one semester will be wrong later. Re-read it annually and check it against your own flag data.

Ready to see which sentences read machine-made?

If you are the person who has to make the call, get the sentence-level view before you get the argument. Run a draft through DeAIze’s AI detector to see the document score and every sentence it flagged, so you know whether you are looking at one odd paragraph or a whole document. If the flag sits on your own writing, the draft editor lets you rewrite only the flagged lines and check whether the meaning survived. Pricing is pay-as-you-go, credits never expire, and the free tier gives you 200 words on signup with no credit card — start at deaize.com/register or check the pricing page first. Detector scores are probability estimates, not proof of authorship. Use them to decide where to look, not what to conclude.