Kpmg's ai study pulled: hallucinations expose deep flaws in agentic ai
A highly touted study from accounting giant KPMG, promising to redefine customer experience through ‘agentic AI,’ has been retracted after revealing a shocking number of factual errors and outright fabrications – a stark warning about the current state of artificial intelligence.
Ai’s dangerous tendency to invent
The report, titled “Total Experience: Redefining Excellence in the Age of Agentic AI,” explored how companies were leveraging AI to personalize customer interactions. However, a meticulous investigation by GPTZero and the Financial Times uncovered a disturbing pattern: widespread ‘hallucinations’ – instances where the AI confidently presented incorrect, fabricated, or nonsensical information. This isn’t mere misinterpretation; the AI was actively creating data, undermining the very foundation of trust.
KPMG’s findings mirrored a growing concern within the cybersecurity community. As AI models predict the most likely word based on statistical probability, they occasionally prioritize fluency over accuracy, leading to these ‘hallucinations.’ Furthermore, poorly trained models, relying on outdated or incomplete data, can ‘guess’ at appropriate responses, compounding the problem.
Examples of ai deception
The report itself was riddled with these issues. KPMG touted the existence of a mobile chatbot, ‘Sara,’ for Emirates Airlines capable of adjusting flight plans – a claim swiftly debunked when Emirates confirmed Sara’s launch was in 2023 and lacked that functionality. Similarly, the study highlighted ‘AI agents’ at Swiss investment bank UBS managing investment advisory, risk, and compliance, a detail UBS immediately refuted, stating the information was “factually incorrect.” Even seemingly minor details, like the Swiss Federal Railways’ (SBB) AI-powered trip planners, proved to be entirely fabricated.
The ramifications extend beyond mere embarrassment. The prevalence of these hallucinations – only five of the 45 cited sources being legitimate – raises serious questions about the reliability of AI-generated insights, actively deterring users from relying on this rapidly evolving technology. Indeed, the investigation revealed that half the report’s claims were entirely unsubstantiated, a testament to the risks involved in deploying nascent AI systems without rigorous validation.
Mitigating the risk: practical steps
Despite the setback, experts suggest strategies to reduce the likelihood of encountering AI hallucinations. Keep prompts concise and specific, providing the AI with direct source material. Assigning a clear role to the AI can also improve accuracy. Utilizing multi-step prompting – asking the AI to think through each step – and adjusting the temperature setting (reducing the model’s tendency to improvise) are proven techniques.
KPMG has since pulled the study, acknowledging the need for a thorough review of its publication process. This incident serves as a crucial reminder: the promise of AI rests on verifiable information, not dazzling illusions. The pursuit of ‘total experience’ shouldn’t come at the cost of truth.”n