How Does AI Detection Work? Inside an AI Text Detector

An explanatory paragraph pasted into a chat window; a few seconds later, the verdict: “Moderate.” What does the machine actually see in those lines? Follow Victor’s reading and you will understand what a detector looks at, and why its verdict never counts as proof.

Victor’s verdict, and the question that follows

On nation.ai, you paste an explanatory paragraph into Victor’s chat window. The answer arrives in seconds.

His reading, in detail: “What I look at.”

• “Depending on the text and its purpose, I may consider:”
• “Structure and rhythm: Are sentences and paragraphs unusually uniform or arranged in a very tidy, predictable sequence?”
• “Repeated patterns: Do ideas, transitions, or sentence shapes recur mechanically?”
• “Specificity and voice: Does the text include concrete, relevant details and a distinctive perspective, or remain consistently generic and impersonal?”
• “How ideas develop: Does it offer meaningful reasoning, or mainly restate familiar points in polished language?”
• “Fit for context: Formal, academic, and practical writing is naturally more structured, so those features carry less weight on their own.”

A caution to close it: “No single feature settles the question. A polished structure, correct grammar, or a list of examples can be entirely human; awkwardness or errors do not prove human authorship either. I also don’t treat automated detector scores as proof.”

Victor is the AI checker of Nation AI: a free trial, no account and no card, and the decisive passages on request.

The words that come too easily

Generated text often picks the most probable words: that is the first clue. In 2019, researchers turned it into a tool, GLTR, which colors each word by how likely a model was to choose it. In their study, the detection of generated text rose from 54% to 72%, with no prior training.

The same idea in 2023 with DetectGPT: the method judges a text with the probabilities of the model itself, without training a separate classifier.

How Does AI Detection Work? The Two Measures Behind It

Two measures carry most of the vocabulary, and both start from the same question: how surprised would a language model be by the words in front of it? Perplexity answers it word by word. A sentence a model would have completed almost automatically has low perplexity, and that flat, foregone quality is what generated prose tends to show. GLTR turned the measure into colors on a page.

Burstiness looks at the same quantity across a whole document: “a measure of how much the perplexity varies over the entire document,” in the words of GPTZero, the tool that popularized both terms. A text that mixes long winding sentences with short blunt ones varies a lot. A text where every sentence keeps the same tempo varies little.

The terms travel further than the tools that coined them. GPTZero says it no longer uses either of them, having moved to a deep-learning architecture in the autumn of 2023, and neither measure comes with a shared scale. Victor’s own account of his reading is not framed that way: he describes structure, rhythm, repeated transitions and specificity, and treats no single feature as decisive.

The classifier OpenAI ended up switching off

From GLTR to the watermark published in Nature, the recent history unfolds in six dates.

1. 2019

   Researchers publish GLTR, which helps spot generated text with the naked eye.

2. January 31, 2023

   OpenAI puts a text classifier online, warning that it is not fully reliable.

3. July 20, 2023

   Six months later, the company withdraws it, a casualty of its low accuracy rate.

4. May 14, 2024

   Google DeepMind announces SynthID, a watermark for texts and videos generated by AI.

5. August 4, 2024

   OpenAI describes its watermarking method and explains why it is not using it for now.

6. October 23, 2024

   The SynthID-Text study appears in Nature; its code is opened up the same month.

The invisible mark slipped into the text

A third path remains: marking the text at its source. SynthID-Text, Google DeepMind’s watermark, changes only the step where the model picks each next word, and detecting it never requires running the model underneath.

The watermark stands up to some transformations, Google explains: cropped pieces of text, a few words changed, or mild paraphrasing. Heavy rewriting weakens it, and the page’s own caution is one line long: “Watermark detection is probabilistic.”

OpenAI keeps its own method in reserve: it stands up less well to translation or a rewording by another model, and the company worried it could “stigmatize use of AI as a useful writing tool for non-native English speakers.”

A watermark, finally, only exists if the model that wrote the text added it while writing.

Why no detector can be certain

Rewording moves the score, not the origin. In the 2023 study behind that card, a recursive paraphrasing attack pushed detection rates down sharply while only slightly degrading the text, and the SynthID watermark itself weakens once a text is thoroughly rewritten. A detector reads wording. Change the wording and the reading changes, without anyone learning who wrote the first draft.

Victor is equally clear about what escapes him:

• “I usually can’t establish who wrote it, whether AI was used, how much it contributed, whether a person rewrote the output, or which model or tool was involved.”
• “Human writing, AI-generated writing, and writing revised by people or AI can overlap substantially in style.”

And he sums up what his read is worth: “So my conclusion is about how the text may be perceived, not its provenance. Short passages are especially hard to assess because they provide fewer stylistic patterns to examine.”

Common questions