Why AI invents references (how citation hallucination works)
In short
AI invents references because a language model generates the most plausible next words, not retrieved facts. It has no built-in index of what has actually been published, so when asked for sources it assembles a citation that matches the statistical shape of one — correct format, real-sounding authors, a well-formed DOI — without any guarantee the work exists. This is called citation hallucination.
How a model produces a citation
A large language model is trained to predict the next token in a sequence. It learns the form of academic writing — including the rigid pattern of a reference — extremely well. When you ask for a source, it continues the pattern: a surname, an initial, a year in parentheses, a title-case phrase, a journal name, a volume and page range, a 10.xxxx/… DOI. Each piece is locally plausible, but the model is composing, not looking anything up.
Why references specifically go wrong
- No retrieval by default. Without a connected database, the model can only draw on patterns from training, not a verifiable record.
- Citations are high-regularity, low-redundancy. Their format is predictable, but the exact author–title–venue combination is not — so the model fills the gap with a confident guess.
- It is optimised to be helpful. Asked for five sources, it would rather produce five plausible-looking ones than admit it has none.
What actually fixes it
The reliable remedy is external verification: check every reference against authoritative databases rather than asking the model to grade its own work. That is exactly what CiteProve automates — resolving identifiers and matching titles and authors against Crossref, OpenAlex, PubMed and CourtListener.
Related: AI-hallucinated citations · How to detect ChatGPT's fake citations
Related questions
- Does turning on web search stop the hallucinations?
- It reduces them but does not eliminate them. Retrieval-augmented answers can still cite a source that doesn't say what the model claims, or blend a real link with an invented citation, so the references still need checking.
- Are some references more likely to be invented than others?
- Yes. Highly specific, niche, or recent citations — where the model has seen few or no real examples — are the most likely to be fabricated, because the model falls back on the general shape of a citation.