AI Detection Tools 2026 — Can You Spot AI Writing?

AI detection tools promise a verdict on a question that sounds simple: did a person or a language model write this? What they actually deliver is a probability based on how predictable the wording is, and that gap between a verdict and a probability has already cost students grades and writers their credibility. A 2023 study led by Stanford researchers found that seven popular detectors flagged more than half of essays by non-native English speakers as machine-written. This article explains how the tools work, what the best evidence says about their error rates, and how to use them without ruining someone's semester.

The short answer: a smoke detector, not a fingerprint scanner

A fingerprint scanner compares your finger with a stored print and either matches or does not. An AI detector works more like a smoke detector in a kitchen. It reacts to a pattern that is often caused by fire, but toast, steam, and a hot pan can trigger it too, and a slow, smoldering fire can stay silent. The pattern is a symptom, and a symptom is not a cause.

That is the most useful mental model for everything below. A detector looks at statistical properties of a passage and reports how closely they resemble the output of language models. It never sees who typed the words, how long they took, or what notes sat beside the draft.

Three points summarize the evidence covered in this article:

• Error rates depend heavily on the tool, the text length, and the writer. Older and free detectors performed poorly in 2023 studies. A 2025 working paper found that three commercial tools kept false positives below 1 percent on its test set.
• Bias is real and documented. Detectors have flagged non-native English writing at alarming rates.
• Evasion is possible. Paraphrasing and rewriting reduce detection, and researchers have shown this for several detector designs.

The practical rule that follows: a detector score can start a conversation, but it should not end one.

How AI detectors actually work

Most detectors use some combination of two ideas. The first is to measure how surprising each word is to a reference language model. The second is to train a classifier (a program that sorts inputs into categories) on large piles of labeled human and machine text. Both approaches have the same blind spot: they learn what text from certain models and certain people looked like in the past.

Perplexity: how surprised is the model by each word?

Perplexity is a measure of how unpredictable a passage is to a language model. If you type "peanut butter and" and the next word is "jelly," the model is barely surprised. If the next word is "gravel," it is very surprised. Text with low average surprise has low perplexity.

Language models generate text by repeatedly picking likely next words, so their output tends to have low perplexity. A human writer picks odd words more often. That is the whole intuition. It is also why sampling settings matter: a model run at a higher temperature (a setting that makes word choices more random) produces more surprising text, which we cover in temperature and top-p sampling.

Burstiness: do the sentences all look alike?

Burstiness measures variation across sentences. A person often writes a long, winding sentence and then a short one. Early model output tended to be more uniform. GPTZero, which Princeton student Edward Tian launched in early January 2023, popularized perplexity and burstiness as the two headline scores, according to NPR and other coverage at the time.

A caution is in order. GPTZero's own product has changed since 2023, and third-party reviews report that it moved to a percentage-style output. Check the vendor's current documentation instead of assuming the 2023 description still applies.

Trained classifiers: learning the look of machine text

Modern detectors usually add a trained classifier. Developers collect human writing and model output, label them, and train a neural network to tell them apart. If you want to understand why such a classifier can be both impressive and brittle, the explanation is in how large language models work: the classifier has learned the habits of specific model families and specific kinds of human text.

When a new model family appears, or a user asks a model for a certain voice, those habits shift. A classifier trained on last year's output has not seen this year's. That lag is a structural weakness, not a bug a vendor can patch once and forget.

What the research found, study by study

Marketing pages quote accuracy figures that no independent party has checked. The studies below are the ones that did the checking. Each is dated, because a result from 2023 does not automatically describe a detector updated in 2025.

Source	Year	What was tested	Main finding
OpenAI AI Classifier	January 2023 (withdrawn July 20, 2023)	OpenAI's own detector on a "challenge set" of English texts	Correctly flagged 26% of AI text; wrongly flagged 9% of human text; withdrawn for low accuracy
Liang et al., Patterns	2023	Seven detectors on 91 TOEFL essays and 88 US eighth-grade essays	Average false positive rate of 61.22% on TOEFL essays; native eighth-grade essays were classified far more accurately
Weber-Wulff et al.	2023	14 detection tools, including Turnitin	Authors concluded the tools are "neither accurate nor reliable"; main bias was labeling AI text as human
Turnitin's stated figures	2023	Turnitin's own description of its detector	Under 1% false positives at document level (for documents with 20% or more AI writing); about 4% at sentence level
Jabarian and Imas, NBER	2025	GPTZero, Originality.ai, Pangram, and RoBERTa on about 2,000 human passages in six genres	Commercial tools under 1% false positives; Pangram near 0; false negatives from 0 to 40% depending on tool and model
Dathathri et al., Nature	2024	SynthID-Text watermark	Improved detectability over comparable watermarks; weakened by paraphrasing, translation, and heavy editing

Sources for each row are listed at the end of the article. The rows do not all measure the same thing, so do not average them. They are better read as a timeline: weak first-generation tools, a documented fairness problem, and then a newer generation that reports much better numbers under specific test conditions.

OpenAI's withdrawn classifier

In January 2023, OpenAI released a classifier for flagging AI-written text. On its own challenge set of English texts, it correctly identified 26 percent of AI-written text and mislabeled 9 percent of human-written text as AI. OpenAI said the tool was unsuitable as a sole decision-maker and was weaker on short passages and on languages other than English.

On July 20, 2023, OpenAI noted that the classifier was no longer available because of its low rate of accuracy and said it was researching more effective techniques for establishing the provenance of text. The tool's creator could not make a reliable general-purpose detector in the early period, and that fact shaped how many universities approached the technology.

The Stanford-led bias study

Weixin Liang, James Zou, and colleagues tested seven GPT detectors on 91 essays written for the Test of English as a Foreign Language (TOEFL) and 88 US eighth-grade essays. Across the detectors, the average false positive rate on the TOEFL essays was 61.22 percent. All seven detectors unanimously flagged 18 of the 91 essays (19.78 percent) as AI-written, and at least one detector flagged 89 of 91 (97.80 percent). The paper appeared in the journal Patterns in 2023.

The authors' explanation is perplexity. Non-native writers often show less lexical richness and less syntactic variety, which produces lower-perplexity text, and a detector that equates low perplexity with machine writing mistakes careful second-language prose for a model's output. In a follow-up experiment, prompting ChatGPT to enhance the word choice of those same essays dropped the average false positive rate from 61.22 percent to 11.77 percent. That result has an uncomfortable implication: the detectors reward fancier vocabulary, not authorship.

The study used a specific set of free-to-use detectors circa early 2023. It does not prove that every detector in 2026 behaves identically, but it established a standard that any vendor should be asked to meet: report error rates broken down by writer background.

The 14-tool comparison

Debora Weber-Wulff and an international team of academic-integrity researchers tested 14 detection tools, 12 publicly available ones plus Turnitin and PlagiarismCheck. They concluded that the tools are "neither accurate nor reliable." Their main finding was a bias toward labeling AI-generated text as human-written, and content obfuscation techniques significantly reduced performance further.

That detail matters for the usual narrative. Public attention focused on innocent students being accused, but the same study shows that detectors also let a lot of machine text through. A tool that makes both kinds of errors is a weak foundation for either catching cheaters or protecting honest writers.

Turnitin's own numbers

Turnitin says its detector has a false positive rate under 1 percent at the document level, but that figure applies to documents with 20 percent or more AI writing. At the sentence level, the company describes a rate of around 4 percent, meaning that a sentence highlighted as AI-written has about a 4 percent chance of being human. Turnitin also reports that false positives cluster in documents mixing human and AI writing, particularly at transitions between the two.

Vanderbilt University did the arithmetic in August 2023. It had submitted 75,000 papers to Turnitin in 2022. At a 1 percent false positive rate, about 750 papers could have been incorrectly labeled as partly AI-written. Vanderbilt also cited the non-native speaker bias, the lack of transparency about how the tool decides, and the difficulty of the task, and it disabled the feature. Its note said AI detection is very difficult, if not impossible, and will get harder as AI tools advance.

The 2025 test of newer commercial tools

The most encouraging result is recent. Brian Jabarian and Alex Imas of the University of Chicago built a dataset of about 2,000 human-written passages across six genres (blogs, consumer reviews, news articles, novels, restaurant reviews, and résumés) and generated matching passages with four large language models. They tested three commercial detectors (GPTZero, Originality.ai, and Pangram) and one open-source model (RoBERTa).

All three commercial tools kept false positives below 1 percent, with Pangram's essentially zero at most thresholds. False negatives, where AI text slips through, ranged from roughly 0 to 2 percent for GPTZero, 2 to 4 percent for Pangram, and 10 to 40 percent for Originality.ai depending on the model that wrote the text. The open-source RoBERTa detector performed far worse and was called unsuitable for high-stakes use. Accuracy fell for all commercial tools on passages under 50 words.

Read this result carefully. It is a working paper, not a final peer-reviewed article, and it tests a defined set of genres and models. The authors themselves describe the field as a technical arms race with "humanizer" tools that rewrite AI text to evade detection. It shows that some commercial detectors improved a great deal since 2023. It does not show that a score on a particular student's essay is proof, and it did not test the non-native writer scenario that Liang et al. flagged.

Why a low error rate can still produce a lot of wrong accusations

Even a detector with a very low false positive rate produces wrong flags when most of the people being checked did nothing wrong. This is the base rate problem, and it is the same logic that makes a medical screening test for a rare disease return many false alarms. It is explained in more depth in our piece on why 99 percent accuracy can be useless.

Here is an illustrative calculation with made-up round numbers, not data from any vendor. Suppose a class submits 1,000 essays, and a detector catches 80 percent of AI-written essays while wrongly flagging 1 percent of human-written ones.

Scenario	AI-written essays	Correctly flagged	Human essays wrongly flagged	Share of flags that are wrong
100 of 1,000 essays used AI	100	80	9 (1% of 900)	about 10%
10 of 1,000 essays used AI	10	8	about 10 (1% of 990)	about 55%

In the second scenario, more than half of all flagged essays belong to honest students, even though the detector looks excellent on paper. The less common cheating is, the worse the problem gets. And if the false positive rate for a subgroup is much higher, as Liang et al. found for non-native writers, those students absorb a disproportionate share of the harm.

This is why an instructor cannot treat "the tool said 95 percent AI" as the end of an inquiry. The number is a probability about the text under the tool's assumptions, not a probability that the student cheated.

Why detectors fail: five structural reasons

The failure modes are worth separating, because each suggests a different response.

1. The target keeps moving

Detectors are trained on output from particular models. When providers release new models, or when users ask for a different voice, the statistical habits change. The classifier has to be retrained, and during the gap it is guessing.

2. Rewriting breaks the signal

Sadasivan and colleagues argued, with both experiments and theory, that detectors can be defeated by recursive paraphrasing, in which a second model rewrites the output repeatedly. Their paper also made a theoretical point: the best performance any detector can achieve is bounded by how far apart the human and machine text distributions are, so the closer models get to human text, the harder reliable detection becomes. The Weber-Wulff team reached a similar practical conclusion from the other direction, finding that obfuscation reduced tool performance significantly.

3. Fluent, formulaic, and careful writing looks machine-made

Detectors lean on predictability. Predictable writing includes second-language prose, formulaic lab reports, legal boilerplate, and anything heavily edited for clarity. Liang et al. documented the second-language case. Turnitin's own description that false positives rise at human and AI transitions suggests that the mixed-authorship case is also hard.

4. Short passages carry little signal

Every statistical test needs enough data. OpenAI said its classifier was less reliable on short text, and the Chicago study found accuracy fell for all commercial tools on passages under 50 words. A single paragraph, a discussion post, or a short answer simply holds too little evidence for a firm conclusion.

5. Most real writing is mixed

A student might brainstorm with a chatbot, write a draft alone, ask a model to tighten two paragraphs, and then edit the result by hand. Which parts are "AI"? A detector forced to give one number for a gradient of human and machine contribution is answering a question that has no clean answer. Policies matter more here than software, and the best-run schools write rules about acceptable use instead of relying on a classifier to infer it.

What watermarking can and cannot do

If detecting text after the fact is hard, an alternative is to mark text when it is generated. A watermark changes the model's word choices in a subtle statistical pattern that a detector with the right key can later recognize. Think of it as a faint paper watermark in currency rather than a guess about the paper's feel.

Google DeepMind's SynthID-Text, described in a 2024 Nature paper by Dathathri and colleagues, is the most prominent deployed example. It modifies only the sampling step, so it does not affect model training, and detection does not require access to the underlying model. In a live test covering nearly 20 million Gemini responses, users showed no significant preference between watermarked and unwatermarked outputs, which suggests the quality cost is small.

The limits are just as clear. Paraphrasing, translation, and heavy editing weaken the watermark, and it only works for text from providers who adopt the same scheme. A student who uses a model without a watermark, or an open model run privately, is invisible to it. Researchers have also argued that watermarks can be spoofed, so that human text is made to look watermarked. Watermarking helps with cooperative providers at scale. It is not a general answer for a single classroom.

What detectors catch and what they miss

Putting the evidence together, a realistic picture looks like this.

Situation	Detector reliability	Why
Long passage, unedited output from a mainstream model	Highest	Strong statistical signal, matches training data
Long passage, light human editing	Moderate	Some signal remains, but errors rise
Passage run through a paraphraser or "humanizer"	Low	Obfuscation reduces performance in independent tests
Short text under about 50 words	Low	Too little data; accuracy fell in the 2025 test
Non-native English academic writing	Risk of false positives	Documented bias in the 2023 study
Mixed human and AI drafting	Unclear	Errors cluster at transitions, per Turnitin

The table is a summary of the studies above, not a measurement of any specific product. If you are choosing a tool, ask the vendor for results on text like yours, published error rates by writer background, and a clear statement of what the score means.

Better approaches than a score

Institutions that have thought hard about this tend to converge on a few practices.

Evaluate the process, not only the product

Ask for outlines, drafts, notes, and source lists along the way. Revision history in a word processor shows how a document grew. A final essay that appears fully formed with no history is a reason for a conversation, and a conversation is a far better instrument than a percentage.

Use assessments that need the student in the room

Short oral follow-ups, in-class writing, and discussion-based evaluation test understanding directly. A student who wrote an essay can explain choices in it. This is also kinder: it gives a falsely accused student a way to demonstrate authorship.

Write clear policies about AI use

Some courses ban AI, some allow it with disclosure, and some require it for specific tasks. Students cannot follow a rule they have not been given. For teachers designing these policies, our overview of AI tools for teachers covers the other side of the classroom.

Treat disclosure as a skill

Many educators now ask students to state which tools they used and how, and to critique the model's output. That turns a hidden behavior into something that can be graded. It also tests whether the student can spot an error, which relates directly to why AI hallucinates facts.

If you must use a detector

• Use it only to decide whether to look closer, never to decide guilt.
• Test it on writing you know the origin of, including work by non-native speakers, before relying on it.
• Tell students it is in use and what happens after a flag.
• Keep a human who can overrule the score, and write down the other evidence considered.
• Re-check performance when new models appear.

What this means for writers and editors

If you write for a living, a detector may eventually flag your work. That says something about the tool's assumptions and little about your honesty. Keep drafts and notes, because that paper trail is the best defense against a bad flag.

If you use AI in your process, the more important question is quality and accuracy, not whether a detector can see it. Fluent text that is wrong is still wrong. Publishers who care about reader trust, a topic we examine in how to spot AI-written science fluff, usually find that vague, unsourced, and generic writing is the real tell, and it can be spotted by a careful reader without any software. For guidance on judging writing assistants themselves, see how to evaluate AI writing tools without hype.

Where detection is likely heading

Nobody can predict this field with confidence, but the evidence suggests a few directions. Detectors will keep improving on unedited output from mainstream models, as the 2025 results show. Evasion tools will keep improving as well, so a good score today carries no guarantee for tomorrow. Provenance tools such as watermarking and signed metadata will matter more for large platforms than for individual classrooms. And institutions will keep moving from policing output to assessing understanding, because that shift does not depend on winning an arms race.

For comparison shopping among the chatbots themselves, our ChatGPT vs Claude vs Gemini comparison covers the generation side.

Two companion pieces cover the rest of the picture: best ChatGPT alternatives for the wider tool field, and AI writing tools for bloggers for the commercial side.

How to read a vendor's accuracy claim

A detector's marketing page usually leads with a single percentage. That number is almost meaningless without context, and a handful of questions will tell you how much weight it deserves.

What was the test set? Accuracy on clean essays by native speakers says little about a classroom that includes second-language writers, lab reports, or heavily edited drafts. The 2025 Chicago study used about 2,000 human passages in six genres, which is more varied than many vendor tests, but it did not test second-language writing specifically. Ask whether the vendor reports results for the kind of writing you will actually check.

Which error is being reported? A tool can post a high "accuracy" number while making many false positives, many false negatives, or both. The two errors have very different costs. A false positive can wrongly accuse a person. A false negative lets machine text through. Insist on both rates separately, plus the threshold used to calculate them. The Chicago authors proposed comparing tools under a strict cap on false positives (they used 0.5 percent as an example), which is a useful standard because it forces the comparison onto the error that harms people.

At what level is the error measured? Turnitin's own figures show why this matters. Its document-level false positive rate of under 1 percent applies to documents with 20 percent or more AI writing, while its sentence-level figure is around 4 percent. A highlighted sentence is a much weaker claim than a flagged document, and a teacher who reads highlights as proof is using the weaker number as if it were the stronger one.

How old is the evidence, and which models were tested? Detectors are tuned against particular generations of language models. A 2023 result says little about a 2026 model, and a result from last year may not describe the version that is running today. Look for a date and a model list.

Who did the testing? Independent academic tests, such as the Weber-Wulff team's 14-tool comparison or the Chicago working paper, carry more weight than a vendor's internal benchmark, because the people running them have no sales target. Even independent studies can be limited by their sample, so the right move is to look for agreement across several of them instead of leaning on one.

What a fair process looks like after a flag

If an institution or employer does use a detector, the process around the score matters as much as the score. A fair process has a few features, and each of them follows directly from the research above.

First, the score opens an inquiry and does not conclude one. Because even a very good detector produces mostly wrong flags when real misuse is rare, as the base rate table showed, a flag is a prompt to gather evidence.

Second, the person flagged gets to respond with their own evidence: drafts, notes, browser history of sources visited, and the revision timeline in the document. A student who actually wrote the essay can usually reconstruct how it grew.

Third, someone with subject knowledge reads the work. Stylistic oddities, claims the student cannot explain, and citations that do not exist are more telling than any percentage, and they are signals a human reviewer can weigh in context.

Fourth, the institution records which detector, which version, and which threshold it used, so that the decision can be reviewed and the tool's performance can be audited over time. The Chicago authors recommend regular, published performance audits for exactly this reason.

Fifth, the policy tells students in advance what is allowed. A rule that says "do not use AI" with no definition leaves a student who used a grammar checker, a translation tool, or a brainstorming prompt guessing about where the line sits.

FAQ

Q: Are AI detectors useless? A: No, but their usefulness is narrow. A 2025 working paper found that three commercial tools kept false positives below 1 percent on its test set, and detectors can flag unedited machine output. They remain unreliable as proof, especially on short, edited, mixed, or non-native writing.

Q: Why did OpenAI shut down its own detector? A: OpenAI said the classifier was no longer available as of July 20, 2023, because of its low rate of accuracy. At launch it correctly flagged 26 percent of AI text and mislabeled 9 percent of human text on a challenge set of English texts.

Q: Do detectors discriminate against non-native English writers? A: A 2023 study in Patterns found that seven detectors misclassified TOEFL essays by non-native writers as AI-generated at an average rate of 61.22 percent. The authors attribute this to lower perplexity in such writing. Newer tools may differ, so ask vendors for results by writer background.

Q: Can paraphrasing tools defeat AI detectors? A: Often, yes. Weber-Wulff and colleagues found that obfuscation significantly reduced detector performance, and Sadasivan and colleagues showed recursive paraphrasing could defeat several detector designs. Results vary by tool, so a passing score is not proof of human authorship either.

Q: What should I do if a detector wrongly flags my work? A: Share your drafts, notes, sources, and revision history, and offer to discuss the work in detail. Point out that detector scores are probabilities with documented error rates, and cite the Vanderbilt decision to disable Turnitin's detector if the institution treats the score as conclusive.

Q: Should teachers use AI detection at all? A: Some institutions do, and some do not. If used, treat a flag as a prompt for a conversation and gather other evidence first. Vanderbilt chose to disable Turnitin's AI detector in August 2023, citing false positives, bias concerns, and lack of transparency.

Q: Will watermarking solve the problem? A: Not by itself. SynthID-Text improves detectability for text from providers that adopt it, but paraphrasing, translation, and heavy editing weaken it, and it cannot flag text from models that do not use the same watermark.

The takeaway

An AI detector measures resemblance to machine text, and resemblance is not authorship. The evidence shows a field that has improved since the failures of 2023, with some commercial tools reporting very low false positive rates in a 2025 test, while still struggling with short passages, paraphrased text, mixed drafting, and the writing of non-native English speakers. The base rate problem means that even good detectors will accuse some honest people whenever honest people are the majority. Use a score the way you would use a smoke alarm: as a reason to check the kitchen, not as proof of arson.

Sources

• Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., and Zou, J. "GPT detectors are biased against non-native English writers." arXiv:2304.02819, published in Patterns, 2023.
• Weber-Wulff, D., et al. "Testing of Detection Tools for AI-Generated Text." arXiv:2306.15666, 2023.
• Vanderbilt University. "Guidance on AI detection and why we're disabling Turnitin's AI detector," August 16, 2023.
• Turnitin. "Understanding the false positive rate for sentences of our AI writing detection capability."
• Jabarian, B., and Imas, A. "Artificial Writing and Automated Detection," NBER Working Paper 34223, 2025, summarized in Chicago Booth Review, December 2025.
• Sadasivan, V. S., et al. "Can AI-Generated Text be Reliably Detected?" arXiv:2303.11156.
• Dathathri, S., et al. "Scalable watermarking for identifying large language model outputs." Nature 634, 818-823 (2024), code and details.

Last verified: October 20, 2026