In brief
The Supreme Court’s October survey of 360 judges documents that about 80 percent have witnessed AI-generated fake precedents in civil, family, and administrative litigation. Almost all respondents urged regulatory measures, including fines and verification protocols, to curb the misuse of artificial intelligence in court documents. The findings underscore an emerging legal‑tech challenge as AI models increasingly produce hallucinated legal references.
A Supreme Court survey revealed that eight in ten South Korean judges had encountered court cases citing AI-generated false or nonexistent legal precedents, highlighting a growing problem of artificial‑intelligence hallucination in Korean law. Published on November 2 by Yonhap, the study surveyed 360 judges between October 6–16 and found that 80.7 percent had seen nonexistent citations while 65.4 percent had witnessed false precedents used in legal arguments.
Survey Methodology and Key Statistics
The survey was administered to judges presiding over civil, family, and administrative matters, the most common arenas where AI‑generated content is introduced by attorneys. Judges reported specific instances in which the AI produced references to legal materials that did not exist in any official database or published record. The findings broke down as follows: 164 judges admitted to having cited false precedents themselves, and 136 judges reported that lawyers had presented such citations on their behalf.
In addition to precedents, 16.9 percent of judges—41 respondents—had encountered cases involving AI‑generated false laws or regulations, and 4.1 percent—10 judges—reported AI‑delivered falsified academic theses and publications being used in legal pleadings.
Almost all participants—92 percent—stated that institutional measures are required to address the issue. The survey participants suggested a combination of legal fines, mandatory verification of AI‑generated citations, and disciplinary actions against attorneys who repeatedly cite fabricated information.
AI Hallucination in the Legal Context
The term "AI hallucination" refers to situations where an artificial intelligence model presents false or fabricated information as if it were factual. In the context of Korean law, the phenomenon has manifested as attorneys submitting citations to precedents that cannot be traced to any court record or official publication. Scholars and technologists define these hallucinations as errors arising when large language models rely on pattern recognition rather than a verified knowledge base.
“The presence of AI hallucination in legal documentation threatens the integrity of judicial processes,” warned an official from the National Court Administration. The official added that such fake references can cause procedural delays, undermine public trust, and potentially prejudice case outcomes. “We must develop effective countermeasures based on these survey results,” the official said.
Legal experts note that the Supreme Court’s call aligns with international trends that see courts grappling with the integration of generative AI. In Europe, for instance, the European Court of Justice has begun drafting guidelines on the responsible use of AI in legal analysis. In Korea, the survey underscores a domestic need to codify rules that enforce accuracy and accountability in AI‑assisted legal research.
Proposed Institutional Measures
Respondents proposed several concrete steps to mitigate the problem. First, establishing a legal framework that allows for fines against attorneys who present AI‑generated falsified citations. Second, instituting a verification requirement whereby all AI‑produced citations must be cross‑checked against official databases before being admitted in court. Third, implementing disciplinary mechanisms that target repeat offenders, potentially ranging from monetary penalties to suspension of law‑firm licenses.
These proposals reflect a broader shift toward embedding technical safeguards within the legal profession. By requiring verification, judges would be able to independently confirm the authenticity of citations, thereby reducing the likelihood that AI hallucinations advance in court. Additionally, the proposed fines and disciplinary actions function as deterrents, signalling that misuse of AI tools is a serious professional breach.
While the survey does not provide a single definitive solution, it offers a clear path forward for Korea’s judiciary. It brings to light the tension between the rapid deployment of generative AI and the long-established requirements for documentary accuracy in legal adjudication. If Korea implements the measures suggested, it could serve as a model for other jurisdictions that are rapidly integrating AI into their legal systems.
As the legal landscape continues to evolve, the Supreme Court’s findings underscore the importance of vigilance and proactive regulation. The convergence of AI technology with law may usher in unprecedented efficiencies, but it also demands rigorous oversight to preserve the factual basis upon which justice is built.
