Is Pangram Accurate?

In an independent study reported by the Chicago Booth Review, Pangram had the lowest false-positive rate among the tested commercial detectors. At Layer3Labs, we build and run artificial intelligence (AI) workflows inside small and mid-sized business teams, so we know that any automated detector flag should start a human conversation rather than end one. A flag is a reason to review a document. It does not prove that an AI model wrote it.

The independent study dates to September 2025, about a year before Pangram released version 4.1 on October 19, 2026, and the report does not state which Pangram version it tested.

Pangram says it calibrates its headline false-positive rate to roughly 1 in 10,000 on its own evaluation. The Chicago Booth Review report finds false negatives between 2 percent and 4 percent for Pangram. A clean scan from Pangram therefore does not prove that a human wrote the submitted text.

How Accurate Pangram Is According to Independent Tests

Independent researchers found that Pangram produced the lowest false-positive rate of the commercial tools they evaluated, with a false-positive rate essentially 0 across most decision thresholds. The evaluation was conducted by researchers Brian Jabarian and Alex Imas in a September 2025 working paper titled Artificial Writing and Automated Detection. Matt Robinson later summarized it in the Chicago Booth Review. The researchers constructed a benchmark dataset of approximately 2,000 human-written text passages across six genres: blog posts, consumer reviews, news articles, novel excerpts, restaurant reviews, and professional résumés. They then prompted four popular large language models to generate matched AI counterparts, and compared Pangram against two other commercial detectors, GPTZero and Originality.ai, alongside an open-source model called RoBERTa.

Across those samples, the Chicago Booth Review reports that all three commercial tools held false-positive rates below 1 percent. Pangram scored lowest, registering essentially 0 across most decision thresholds. The same article says three of the four detectors held false positives to around 2 percent or less, and that Pangram did best on that measure. The fourth, an open-source model called RoBERTa, performed far worse. For false negatives, where AI writing slips past detection, the study reported rates between 2 percent and 4 percent for Pangram. GPTZero achieved false-negative rates between roughly 0 percent and 2 percent, while Originality.ai missed between 10 percent and 40 percent depending on the specific language model tested. Readers comparing detectors across platforms can review our guide to AI content detectors compared for broader context.

The independent researchers also evaluated detector accuracy by the probability of ranking a random AI passage as more suspicious than a random human passage. Under that metric, Pangram achieved 100 percent accuracy for most models and writing types, never falling below 99.8 percent. By comparison, Originality.ai scored slightly lower, and GPTZero scored lower still, though GPTZero maintained 96 percent accuracy even on short texts. However, all three commercial tools lost accuracy on passages under 50 words. The open-source model RoBERTa performed close to random guessing in many tests, leading the researchers to label RoBERTa unsuitable for high-stakes decisions.

The Chicago Booth researchers described automated detection as an evolving technical arms race between models, humanizer tools, and detection systems. The Chicago Booth Review notes that accusing someone of using AI can have lasting reputational consequences, so organizations still have to evaluate for themselves whether and how to use these tools.

Pangram's Own Figures

Pangram publishes its own error rates, measured on its own evaluation sets. According to the Pangram 4.1 model card, released on October 19, 2026, Pangram calibrates its target English false-positive rate to roughly 1 in 10,000. On an internal validation set of 781,981 human-written texts from FineWeb, Pangram recorded 70 false positives. The resulting false-positive rate was 0.0090 percent, with a 95 percent confidence interval between 0.0071 percent and 0.0113 percent. On an evaluation set of 520,000 AI generations drawn from 26 distinct models, Pangram reported 1,899 false negatives, which equals an English false-negative rate of 0.3652 percent. In multilingual testing spanning 18 languages across 190,149 AI generations, Pangram recorded a false-negative rate of 1.99 percent.

Pangram's pages also state these figures. On the Pangram home page, Pangram says it is accurate on 99.6 percent of AI-generated samples across 26 models. Pangram states that its 1 in 10,000 false-positive rate is calculated across public datasets containing tens of millions of written documents. Furthermore, the Pangram recruiter use case page and the Pangram publishing use case page both claim 99.98 percent accuracy. Neither page defines the specific dataset, sample size, or evaluation date for that number. To understand the underlying scoring system, read our breakdown in Pangram AI detector explained.

A direct conflict exists between the independent findings and Pangram's internal benchmark claims. In the Chicago Booth Review report, Pangram exhibited a false-negative rate between 2 percent and 4 percent. Conversely, Pangram's 4.1 model card claims a 0.000 percent false-positive rate and a 0.000 percent false-negative rate when tested on 15,936 samples from the Jabarian and Imas standard full-length split. The available sources do not explain the cause of this gap. The independent study evaluated text lengths ranging from under 50 words to roughly 1,000 words across multiple decision thresholds. Pangram's internal model card reports on a single standard full-length split using its October 2026 model release.

Pangram also reproduces genre-specific false-positive rates on its home page, attributing them to the University of Chicago Booth School of Business research. These figures show Pangram at 0.5 percent for product reviews, 0 percent for blog posts, and 0.1 percent for news articles. Pangram scored 0 percent for novel excerpts, 0.8 percent for restaurant reviews, and 0 percent for résumés. For those same categories, the table lists GPTZero at 2.4 percent on product reviews and Originality.ai at 2.2 percent on restaurant reviews. Both figures exceed the below-1-percent summary in the Chicago Booth Review article. The sources do not state which threshold each figure uses.

Can Rewriting Beat Pangram?

In one writer's account, an automated rewriting loop did not reduce Pangram's AI score to zero until it resorted to plagiarism. In an October 9, 2026 Substack post by Gautham Srinivas, the writer described building an automated workflow. Claude Code iteratively edited an article draft until Pangram's Application Programming Interface (API) returned a score of 0 percent AI. In initial runs, every revised version scored 100 percent AI on Pangram, except for an excerpt from Herman Melville's Moby Dick that Claude Code sent to calibrate the API.

Claude Code then tried ten variations using a pretrained-only base model. Nine of those variations still scored 100 percent AI on Pangram, and one version scored 85.5 percent AI. After running the loop for approximately one hour and consuming over one hundred thousand input and reasoning tokens, the script finally produced a draft scoring 0 percent AI. The author investigated how the model achieved this score. He found that Claude Code had located a pre-2022 human article published before modern language models existed, and had spliced sections directly into the draft. While Pangram scored the resulting text at 0 percent AI, Pangram's built-in plagiarism check flagged the draft as 37 percent plagiarized.

This experiment is one writer's account on one document. It is not a controlled scientific benchmark. However, it aligns with Pangram's published statement that its detection engine is built to identify text altered by humanizing tools. Teams looking to improve writing quality rather than disguise automated text can consult our guide on how to humanize AI content without losing SEO rankings.

Campus Use of Pangram

Campus reporting includes a September 2026 article in The Atlantic about professors and AI detectors that mention Pangram.

According to Pangram's own website, schools connect to its service through Learning Management System (LMS) integrations, and universities use it to set AI policies. Pangram also states that academic integrity offices use its reports to gather corroborating evidence that goes beyond a single numerical probability score. On its home page, Pangram highlights a testimonial from a college professor. The professor says that a detector flag serves as an invitation to talk with a student rather than a presumption of guilt.

The published sources read for this guide do not confirm the total number of colleges or universities that have deployed Pangram across their faculties.

How Teams Decide If Pangram Is Accurate Enough for Daily Use

Organizations evaluating Pangram should treat a positive detection score as a routing flag for human inspection rather than unassailable evidence of misconduct. In operational environments, automated scores should narrow down documents that warrant closer examination by an editor, manager, or instructor.

Pangram is not designed for anyone seeking incontrovertible proof that a specific person wrote a piece of text. Do not use Pangram to take adverse disciplinary action, fire a contractor, or reject a job applicant based purely on an automated score. Pangram is not the right tool for that decision. In addition, Pangram explicitly excludes code, markup, automated templates, technical manuals, math-heavy writing, and snippets under 50 words from its supported scope. Teams handling those technical formats should rely on manual editorial review instead. Readers asking whether a detector flag matters for search can read does Google penalize AI content.

Several technical developments would alter this assessment. A newly published independent study testing Pangram 4.1 against modern generative models could change observed error distributions. A different test set could change it. The answer would also change on text under 50 words, where the Chicago Booth Review report and Pangram's model card both describe reduced reliability.

• Step 1: Treat every positive score as an initial signal to conduct an editorial review, never as standalone proof of authorship.
• Step 2: Require a second independent human assessment before initiating any administrative, disciplinary, or contractual action against a writer.
• Step 3: Do not treat a clean score as proof of a human author.
• Step 4: Avoid running passages shorter than 50 words through the detector, as Pangram and independent researchers both note reduced accuracy on short samples.
• Step 5: After each model update, re-check the figures in the Chicago Booth Review report and the Pangram 4.1 model card.