CITATION REVIEW / CONTROLLED CASE STUDYWhat a citation check can tell you: eight reproducible cases
We ran the same synthetic draft through BOZOS over HTTP and MCP. Both returned the expected eight outcomes, including a PDF passage and explicit uncertainty for incomplete text.
Textual presence, source readability, and unresolved uncertainty are separate findings.
Observed results
| Case | HTTP | MCP | Interpretation |
|---|---|---|---|
| Exact quotation | exact_match | exact_match | The 24-sample sentence appears literally in the text fixture. |
| Whitespace changed | normalized_match | normalized_match | A newline and repeated spaces are normalized; the original matched passage is retained. |
| Curly quotation marks | not_found | not_found | The fixture uses straight quotation marks. Punctuation is not normalized. |
| PDF passage | exact_match | exact_match | The quotation is found on page 2 of a two-page PDF text layer. |
| Partial supplied text | source_truncated | source_truncated | The missing sentence cannot be ruled out because source_truncated is true. |
| Missing source URL | source_unavailable | source_unavailable | The deliberate 404 leaves its quotation unchecked. Other source failures can have different causes. |
| Changed number | not_found | not_found | The quote says 25 samples; the fixture says 24. No literal match was found. |
| Link without a quote | link_readable | link_readable | The source is readable. There is no quotation to match. |
A controlled demonstration
Captured on 25 October 2026 at 08:07 UTC, against BOZOS runtime 1.9.1. The input is an explicit citation manifest tied to a short draft. Seven entries include quotations; one only links a source. The sources are a small text fixture, a two-page PDF, a deliberately missing URL, and caller-supplied partial text.
Both transports returned three quotation matches, two quotations not found, one inconclusive result, one unreadable source, and one readable unquoted link. These are eight deliberately chosen cases, not a representative accuracy benchmark, independent evaluation, or estimate of real-world success rates.
The useful distinction is what remains unchecked
An altered number and altered punctuation both produce not_found in complete checked text. That does not establish that the draft’s claim is false: source versions, extraction, and wording still need review.
The incomplete supplied source produces source_truncated with matched: null. Its absent sentence may exist outside the supplied excerpt. The deliberately missing URL instead reports an HTTP 404 and leaves quote_check null because no source text was read.
The PDF result includes page_start: 2 and page_end: 2, plus the matching text, offsets, and a hash. The whitespace result retains the original newline and spaces in its passage. Matching results identify the words that appeared; a reviewer must still assess context and whether those words support the draft’s claim.
Reproduce the run
Download and unzip the integration kit. From its directory, run the commands below with Python 3.10 or newer. Each output directory must be new. The verifier checks all eight outcomes, the PDF page, missing-source status, supplied-text uncertainty, and agreement between the two transports. It exits with an error on a mismatch.
The archive includes the captured reports and the source fixtures. The published capture records runner and fixture hashes. Report timestamps and live availability can change on a rerun. HTTP and MCP use the same checker, so agreement between them is a transport consistency check, not independent confirmation of correctness.
python3 review.py citations.json --output rerun-http
python3 review.py citations.json --transport mcp --output rerun-mcp
python3 verify_cases.py rerun-http/report.json rerun-mcp/report.json
python3 -m unittest -v test_review.pyLimits of these results
These fixtures are owned by BOZOS, contain extractable text, and are deliberately simple. They do not measure behavior across independent publishers, scanned PDFs, complex layouts, browser-only content, or changing access restrictions. The partial-text case uses caller-supplied text explicitly marked incomplete; it does not exercise a fetched document hitting the reader’s size limit.
There is no OCR, browser rendering, paraphrase matching, truth assessment, or source-credibility scoring. A failed retrieval can reflect access restrictions or extraction limitations. A matched quotation can still be misleading in context. Preserve the full evidence and review the source before relying on a claim.
All fixture material, code, and captured reports are CC0. The text describes a fictional trial solely to exercise matching behavior; no real experimental finding is claimed.