This Week's Term: AI Hallucination - when an AI model confidently generates false information that sounds plausible but has no basis in reality, essentially making things up rather than admitting uncertainty.
In October 2025, Deloitte had to revise a $440,000 government report and issue a partial refund. Their AI confidently cited academic sources that didn't exist and quoted court judgments that were never made. This wasn't a one-off mistake - it's a fundamental feature of how AI works.
While we expect AI to either get things right or admit when it doesn't know, hallucinations represent something different: the AI filling in the blanks with plausible-sounding fiction. The model doesn't "realize" it's making things up - it's simply predicting the most likely next words based on patterns it learned, whether those patterns reflect reality or not.
The stakes are significant. Even the best AI models hallucinate at least 0.7% of the time. In legal contexts, that rate jumps to 6.4%. Air Canada learned this the hard way when their chatbot invented a bereavement fare policy that didn't exist - and the company was ordered to honor it. The industry knows this is serious: $12.8 billion has been invested to address hallucinations, with 78% of AI labs treating it as a top priority.
Most leaders assume hallucinations are a bug that will eventually be fixed. They're not. According to mathematical theory (Gödel's Incompleteness Theorems), AI systems fundamentally cannot be both completely consistent and complete. LLMs are "probabilistic storytellers" predicting the most plausible next word, not truth-seeking engines. As IMD Business School puts it: "LLMs will hallucinate forever."
What's changed is how we're addressing this reality. Rather than eliminating hallucinations, AI labs are teaching models to say "I don't know." OpenAI's GPT-5 now abstains from answering 50% of the time, up from just 1% - a fundamental shift from "always confident" to "appropriately uncertain."
For business leaders, this means stop treating AI as an oracle. Start treating it as what IMD calls a "designated dissenter" - a tool that generates alternatives requiring critical evaluation. The companies that win won't have the most powerful AI; they'll have the most sophisticated human-AI symbiosis, with clear processes for when to trust AI outputs and when to verify.
PwC calls this "tech-powered, human-led" - AI can accelerate work, but humans must verify high-stakes decisions. The key insight: the future belongs not to those with the best AI, but to those who build the best human-AI collaboration.
If you are curious and want to dive deeper, I recommend the Today in Tech episode "Why AI still hallucinates — and how to stop it" featuring Byron Cook, Distinguished Scientist and VP at AWS. Cook explains what hallucinations really are, why they're not always bad, and how automated reasoning can serve as a "logic cop" for generative AI. It's 35 minutes but packed with business-relevant insights on addressing this challenge.