# Known limitations of the Scholar Sidekick citation verifier _v1 — last reviewed 2026-05-26._ The verifier is deliberately scoped and conservative. This page lists the cases where it is weak or can be wrong, so you can judge its output appropriately. It is the companion to the measured numbers and the immutable validation fixture at . These are the edges we currently know about. Found another? See _How to report a failure_ below. ## What the verifier actually checks Given a _claimed_ citation and an identifier (DOI, PMID, PMCID, ISBN, arXiv, ISSN, ADS bibcode, or WHO IRIS URL), it asks: does the claimed **title** — and, as secondary signals, the **first author**, **year**, and **container** — correspond to the paper that identifier resolves to? When the identifier and the claimed title disagree, it title-searches Crossref, PubMed, and OpenAlex to see whether the described work exists elsewhere (a wrong-identifier "citation error") or not at all. It returns one of `matched`, `mismatch`, `ambiguous`, or `not_found`, each with a confidence of `high`, `medium`, or `low`. ## What it does NOT tell you - **Whether the paper supports the citing sentence.** It confirms the citation points to a real, correctly-described paper — not that the paper's findings back the specific claim it is cited for. That still needs a human. - **Whether the paper has been retracted.** A correctly-cited retracted paper returns `matched`. Retraction is a separate signal — see the Retraction Checker. (The verifier ignores publisher "RETRACTED:"-style title prefixes when scoring, so a retracted paper still matches its original title.) - **Whether the paper is open access.** Separate signal — see the Open Access Checker. ## Current weaknesses (v1) ### 1. Translated or transliterated titles Title comparison is character-level. A citation that gives an English _translation_ of a non-English title, or a different romanization of a Cyrillic, Arabic, or CJK title than the registry holds, can score low and surface as `mismatch` or `ambiguous` even when it is the same work. Diacritics and common typographic variants are normalized away; genuinely different words (a translation) are not. _Mitigation:_ cite the title in the form the registry holds, or treat a low-confidence mismatch on a non-English work as "needs a human look." ### 2. Online-first vs print year A one-year gap downgrades a match to `medium` confidence; a two-year-or-greater gap pushes it toward `mismatch / low`. Legitimate online-first-then-print delays (and reprints) can therefore lower confidence on a genuine match — and, conversely, a fabricated citation that happens to be one year off can still read as `medium`. _Mitigation:_ read a year discrepancy as a soft signal and check the resolved record's date. ### 3. Preprint vs published version The same work often has both an arXiv ID and a journal DOI, with slightly different titles, author lists, year, and (always) a different container. Verifying a journal citation against the preprint record (or vice versa) can produce a container mismatch (a soft signal that does not change the verdict) and occasionally a year downgrade. ### 4. First author only Author checking compares the first author (with first/last name-order swap handling). A fabrication that keeps the real first author but invents the co-authors is not caught on the author axis — the title comparison is the gate. Initial-first and given-first forms (`"P Giral"`, `"Kristine Sørensen"`), and a bare surname against an unsplit full name, are now handled; group names supplied as the _author_, and collaboration names folded into the _title_, are too — see _Hardened cases_. ### 5. Registry coverage gaps The "does the described work exist elsewhere?" search covers Crossref, PubMed, and OpenAlex. A real work not indexed there — many book chapters, theses, regional or non-English journals, very recent items, and grey literature — may not be found, so a wrong-identifier case can come back `not_found` or `ambiguous` rather than locating the true record. ### 6. Book edition ambiguity (ISBN) Different editions of a book carry different ISBNs but near-identical titles. The verifier may not distinguish editions, so edition/year differences can lower confidence on what is essentially the right work. ### 7. Ahead-of-print metadata churn A citation captured before a record's metadata is finalized (title tweaks, final year, volume/issue assignment) can mismatch the later, finalized record. ### 8. Conference vs journal duplicates A work published both as a conference paper and a journal article (different DOIs, containers, sometimes years) can produce `ambiguous` results depending on which version the citation targets. ## Hardened cases (previously failed, now fixed) These were real failures we found and fixed; each carries a regression test, and they are listed here for transparency: - **HTML markup in titles** (for example `Gaia`) leaked tag letters into the comparison and dragged similarity below the match threshold — fixed 2026-05-13. - **Publisher retraction prefixes** ("RETRACTED: …", "Retraction Note to: …", "WITHDRAWN: …", "Expression of Concern: …") dragged a correctly-cited retracted paper to `ambiguous` — fixed 2026-05-18. The original title now matches while the retraction prefix still shows on the resolved record. - **First/last name-order inversion** from some adapters caused false first-author mismatches — fixed 2026-05-13. - **Too-sparse claims** (a single-word title with no identifier) returned a misleading `not_found / high`; they now return as "not verifiable" with an explanation — fixed 2026-05-13. - **Group- and consortium-authored citations** cited by their collaboration name (for example `RECOVERY Collaborative Group`, `PIOPED II Investigators`), and reference-list author fields that lump the surname with the given name/initials or transpose the name parts, previously produced false `mismatch` / `ambiguous` verdicts. They now match — the group directly when it is the stored author (after normalising a leading "The"), or, when the record stores only the individual authors, confirmed by title + identifier and returned `matched` with a point-of-use caveat (`_provenance.caveats`) noting the group author itself could not be cross-checked against the record — fixed 2026-05-24, with the earlier _medium_-confidence cap replaced by the caveat on 2026-05-25. See the [changelog](https://scholar-sidekick.com/changelog) for the full set. - **Group author folded into the citation title** — a collaboration name prepended or appended to the title field (for example `"NETTER-1 trial investigators. Phase 3 trial of…"`, or `"…on behalf of the NETTER-1 trial investigators"`) dragged title similarity below the match threshold. The affix is now stripped before the comparison — additively, so it can only rescue a correct citation, never let a fabricated title through — fixed 2026-05-25. - **PubMed collective authors** — a group-authored paper resolved via `PMID` whose PubMed record stores the collaboration as a `CollectiveName` is now rendered with the group name intact (previously mangled into a personal name), so it both formats correctly and matches the group-authored citation — fixed 2026-05-25. - **Structured subtitles** — a citation that kept a work's subtitle while the source (Crossref) stored the title and subtitle as separate fields was scored against the title alone and could fall below the match threshold. The verifier now recombines the canonical title + subtitle before comparing — fixed 2026-05-25. - **Initial-first and given-first author fields** — reference lists often store the first author as initials-or-given-name-first (`"P Giral"`, `"MJ Fila"`, `"Kristine Sørensen"`), or as a bare surname against an unsplit resolved full name (`"kim"` vs `"Hongryel Kim"`). These previously read as an author mismatch. The claimed surname is now recognised as a leading-or-trailing token run of the resolved name in either order; the match is additive and recall-safe — fixed 2026-05-26. - **"Collaborators" group authors** — collaboration names ending in `"Collaborators"` (e.g. the GBD `"… Collaborators"` consortium form) were not recognised as group authors. They now are, alongside the existing investigators / group / consortium / collaboration markers — fixed 2026-05-26. ## How to report a failure Found a citation the verifier gets wrong? Email . Confirmed new failure modes are added to this list and, where fixable, become regression tests.