Source: Reason - Volokh Conspiracy

A Georgia trial court has identified and criticized the submission of likely “hallucinated” case law—nonexistent legal precedents—created by AI legal tools and included in filings by a party. The judge highlighted the risks of relying on generative AI for legal research, emphasizing the need for lawyers to verify citations and arguments before presenting them in court.

The ruling serves as a warning that the legal profession cannot blindly trust AI outputs and must exercise due diligence to avoid misleading the court. The court’s comments are expected to prompt increased scrutiny of AI use in legal practice, especially as errors can have significant consequences for justice and professional responsibility.

The case underscores the emerging challenges of integrating AI into legal workflows, calling for stronger safeguards, ethical standards, and possible updates to rules of professional conduct.