Navigating the ghosts in the machine: Liability, governance lessons from recent AI hallucination cases

Large language models are built to predict probable words, not verify facts, making artificial intelligence hallucinations difficult to eliminate. Courts in China, Germany and the U.S. are now confronting the resulting question: Who bears responsibility when those hallucinations cause harm.

Four courts, one hallucination problem

Four recent cases — two from China, one from the U.S. and one from Germany — show how courts are beginning to answer that question. Although the claims and legal frameworks differ, the cases raise several recurring issues: whether AI providers can avoid responsibility by pointing to the AI or positioning themselves as neutral platforms, whether liability turns on eliminating hallucinations or reasonably mitigating their risks, and whether the hallucination caused legally recognized harm.

The Chinese student case. A Chinese AI chatbot gave a student inaccurate information about a university campus and, when corrected by the student, insisted it was right and promised RMB100,000 in compensation if it ended up being wrong.

The student sued seeking compensation for harm based on the chatbot's promise, but the Hangzhou Internet Court dismissed the claim.

The court reached its conclusion through three findings. AI is not a legal person, so its unprompted compensation promise did not bind the provider, the chatbot is a service subject to fault-based rather than strict product liability, and the provider was not at fault given its limitation warnings and hallucination safeguards, while the student showed no actual harm.

Walters v. OpenAI. When journalist Frederick Riehl asked ChatGPT to summarize a lawsuit, it falsely claimed the complaint accused radio host Mark Walters of embezzlement.