A 5-Step Citation Verification Protocol for AI-Generated Research
A structured workflow for validating AI outputs before they reach a filing or client deliverable.
The legal profession has accumulated a growing number of documented cases of AI-generated hallucinations in court filings since 2023 -- from Mata v. Avianca in New York to sanctioned lawyers in the Netherlands. Each instance followed a common pattern: an attorney trusted AI output without adequate verification, and the fabrication was discovered only after submission.
This protocol establishes a structured approach to citation verification that balances thoroughness with practical time constraints. It applies whether you are using ChatGPT, Lexis+ AI, Westlaw AI-Assisted Research, or any other generative tool.
The Reality of AI-Generated Citations
Even purpose-built legal AI tools produce errors at significant rates. Stanford's 2024 study found that Lexis+ AI hallucinated on 17% of queries and Westlaw AI-Assisted Research on 33%. General-purpose models hallucinate on legal queries 69--88% of the time.
These are not occasional edge cases. They are baseline expectations for how these tools perform.
The good news: verification is straightforward. The bad news: there are no shortcuts that maintain professional standards.
Step 1: Flag Every AI-Generated Element
Before verification begins, you need clear identification of what requires checking.
Flagging criteria:
- Every case citation (name, reporter, volume, page)
- Every quotation attributed to a case, statute, or regulation
- Every factual claim about holdings, procedural history, or statutory language
- Every reference to dates, parties, or procedural posture
- Secondary source citations
Practical method: Work from a clean copy of the AI output. Highlight or bracket every verifiable claim. Do not rely on memory to track what came from AI versus your own research.
Time estimate: 2-5 minutes per page of AI output.
Step 2: Verify Citation Existence
Before analyzing whether a citation supports your proposition, confirm it exists.
For case citations:
- Check the full citation in Westlaw, Lexis, or a free alternative like Google Scholar or CourtListener
- Verify the case name matches exactly
- Confirm the reporter, volume, and starting page are correct
- Check that the year corresponds to the actual decision date
For statutes and regulations:
- Access the official code or register
- Verify the section number exists
- Confirm the language matches current law (not a repealed or amended version)
Red flags suggesting fabrication:
- Case name combinations that seem too on-point for your issue
- Reporter/volume combinations that do not exist
- Courts or jurisdictions that seem unusual for the subject matter
- Dates that do not align with the cited court's history
Time estimate: 1-3 minutes per citation using standard research tools.
Step 3: Validate Substantive Accuracy
A citation that exists may still be mischaracterized. AI tools frequently cite real cases but misstate their holdings.
For cases:
- Read the actual holding, not just the headnotes
- Verify any quoted language appears in the opinion verbatim
- Confirm the case has not been overruled, distinguished, or limited
- Check that procedural posture matches the AI's characterization
For statutes and regulations:
- Read the full text of the relevant provision
- Verify definitions that may affect interpretation
- Check for amendments effective after the AI's training data cutoff
- Review any implementing regulations or agency guidance
Specific verification tasks:
- Page-pinpoint citations: Navigate to the specific page and confirm the language exists there
- Quotations: Search for the exact phrase; AI frequently paraphrases while using quotation marks
- Holdings: Read the relevant section of the opinion; do not rely on the AI's summary
Time estimate: 3-10 minutes per citation, depending on complexity.
Step 4: Assess Current Authority
Even accurate citations may represent bad law.
Required checks:
- Run Shepard's or KeyCite on every case citation
- Review for negative treatment (overruled, criticized, distinguished)
- Check for subsequent legislation that supersedes case law
- Verify regulatory provisions remain in effect
Pay attention to:
- Superseding statutes that may have changed the common law rule
- Circuit splits where your jurisdiction differs
- Recent amendments to regulations
- Pending legislation or rulemaking that may change the landscape
Document your findings: Note the date you ran these checks. If significant time passes before filing, re-run them.
Time estimate: 2-5 minutes per case citation.
Step 5: Document Your Verification
Create a record of what you verified and when. This documentation serves multiple purposes: it ensures completeness during the current project, provides protection if questions arise later, and establishes institutional knowledge for supervision and training.
Documentation should include:
- Date of AI query
- Tool used
- Specific verification steps performed for each citation
- Results of currency checks
- Any discrepancies identified and how they were resolved
Format options:
- Research memorandum with verification section
- Spreadsheet tracking each citation
- Annotations on the working draft
- Formal verification checklist
The appropriate level of documentation depends on the stakes. Routine correspondence may warrant minimal documentation; court filings and dispositive motions warrant comprehensive records.
Time estimate: 5-10 minutes to compile documentation for a typical research memo.
Time Investment Analysis
For a research memorandum containing 10 AI-generated citations:
| Step | Time per Citation | Total Time |
|---|---|---|
| 1. Flagging | 30 seconds | 5 minutes |
| 2. Existence check | 2 minutes | 20 minutes |
| 3. Substantive verification | 5 minutes | 50 minutes |
| 4. Currency check | 3 minutes | 30 minutes |
| 5. Documentation | - | 10 minutes |
| Total | ~2 hours |
This represents approximately 115 minutes for 10 citations. For comparison, conducting the same research without AI assistance would likely take 4-8 hours depending on complexity.
The verification time is not wasted -- it is the minimum professional standard for AI-assisted work. Skipping verification is not efficiency; it is negligence.
When to Skip Verification
Never.
This is not rhetorical. Given documented hallucination rates of 17--33% even for specialised legal AI tools, skipping verification means accepting a statistically significant probability of submitting fabricated content.
Some practitioners argue that verification of routine matters can be relaxed. The counterargument: the Mata v. Avianca case involved routine research on statute of limitations -- a well-established area of law where fabrication should have been easily caught.
If a citation appears in work product that leaves your office, it requires verification.
Scaling for Different Work Products
Quick research queries (internal only):
- Steps 1-2 mandatory
- Steps 3-5 proportionate to stakes if information will be acted upon
Client correspondence:
- Steps 1-4 mandatory
- Step 5 recommended
Court filings:
- All five steps mandatory
- Enhanced documentation
- Consider second-reviewer verification for dispositive motions
Published materials (articles, CLEs):
- All five steps mandatory
- Extended currency monitoring through publication
Key Takeaways
- Verification is not optional—hallucination rates of 17-33% make checking mandatory for professional standards
- A structured five-step protocol ensures nothing is missed: flag, verify existence, validate substance, assess currency, document
- Budget approximately 10-15 minutes per AI-generated citation for thorough verification
- Documentation protects you if questions arise and supports supervisory obligations
- The time investment for verification is still substantially less than conducting research without AI assistance
Sources
-
Stanford HAI — "AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries" (2024): The first preregistered empirical evaluation of AI-driven legal research tools, finding Lexis+ AI hallucinated on 17% of queries and Westlaw AI-Assisted Research on 33%. hai.stanford.edu
-
Stanford HAI — "Hallucinating Law: Legal Mistakes with Large Language Models Are Pervasive" (2024): Study finding general-purpose LLMs hallucinate on legal queries 69-88% of the time. hai.stanford.edu
-
Wikipedia — "Mata v. Avianca, Inc." (2023): Background on the landmark case where lawyers were sanctioned $5,000 for submitting ChatGPT-fabricated citations. en.wikipedia.org
-
Damien Charlotin — "AI Hallucination Cases Database" (2025): Global database tracking over 1,000 documented instances of AI-generated hallucinated content in court proceedings. damiencharlotin.com
-
LawNext — "LawDroid Launches CiteCheck AI" (2025): Coverage of automated citation verification tools designed to supplement manual verification workflows. lawnext.com