< Home / Blog / Hallucinations are a major Problem for CX-focused generative AI. - guest blog by Jonas Berggren.

What happened?

Hallucinations are a major Problem for CX-focused generative AI. - guest blog by Jonas Berggren.

Jonas Berggren

Head Of Business Development NE

Tech and tools

AI,

GenAI,

Published on Thu May 15 2025

Updated on Fri Aug 08 2025

4 minute read

Roberto Mata was a passenger on an Avianca Airlines flight in 2019. He claims that he sustained an injury from a serving kart on the flight and that the injury was caused by the negligence of the airline cabin crew. Mr Mata engaged a lawyer to present his case to the court. Steven Schwartz, an attorney with Levidow, Levidow & Oberman and licensed in New York for over three decades, handled Mata’s representation. Mr Schwartz submitted the negligence case to the court and cited six similar cases where the airline had always paid compensation. The only problem was that he had searched for the cases using ChatGPT and ChatGPT hallucinated when creating the search results. None of the cases really existed. Schwartz himself ended up having to defend his own position as he was then charged with wasting court time by submitting false case information.

What happened?

ChatGPT is very smart. It has been trained on billions of connections between different words - in the case of ChatGPT 4, the word connections number over one trillion. But if ChatGPT is asked a question and it is not 100% sure of the reply, sometimes it will insert ‘almost right’ or ‘likely’ information instead of just answering that it doesn’t know. It’s a bit like when you ask a child a question, and they start making up an answer that is wrong because giving any answer is preferable to just saying, ‘I don’t know.’ In his 2023 book ‘The AI-Empowered Customer Experience’, Simon Kriss explains how hallucination works. Simon’s book says: We are asking the AI to write an example crime report for a stolen bicycle but giving it limited information to work with. Let’s see what else it hallucinates to fill in the gaps. Generative AI (gAI) Prompt: Write me an example of what the narrative of a completed crime report might be for the overnight theft of a bicycle. Write it in a formal criminal justice tone of voice. Because we gave the prompt very little information it will hallucinate its own. When I did this in ChatGPT, the hallucinations included that the reporting person was a 43-year-old male who was visibly distressed and that the Trek 500 bicycle had an approximate value of $2,500. None of this is true, nor was it supplied to the gAI engine. What Simon’s book - and the Avianca case - clearly demonstrates is that when a Generative AI tool like ChatGPT is not sure of the next statement, it will often fill in the blanks with the most likely text. Simon didn’t specify the age of the person whose bicycle was stolen, but it may be required information on a crime report so ChatGPT just made some assumptions. This has important ramifications for the use of gAI in the customer service environment. We must appreciate that ChatGPT (or similar AI models) is not just a dictionary of answers. It is not just a big list of Frequently Asked Questions. It is an AI system that is trained on a large amount of data and each answer is generated in real-time - it is not pre-scripted. We should not be surprised about this tendency to hallucinate. Humans do it all the time. If you have never heard of the Simons and Chabris selective attention test, then you can try it for yourself on YouTube here. This is proof that when our eyes are focusing on one task, they might be completely ignoring something else. Our eyes are such detailed cameras that if our brain had to process every single ‘pixel’, it would be impossible ever to get anything done. Musicians will also know that when they are in a flow state, their hands will move to the shape of chords and notes. In fact, this is why learners of the guitar and piano struggle - they are still thinking about each individual finger and where it needs to be.

Leading Voices: most brands fail at agentic AI. Our solutions experts share how yours can win.

Transcom

Created at Thu Sep 17 2026

6 min read

Leading Voices: most brands fail at agentic AI. Our solutions experts share how yours can win.

Something happens when a technology concept outpaces the terminology we have to describe it. Across the customer experience landscape, vendors start applying the same label to wildly different things. For instance, industry leaders start buying into the AI label instead of its real operational capability. And somewhere between the pitch deck and the production environment, the real question gets lost: does any of this actually address a real operational need? Agentic AI has reached that stage.

Hallucinations are a major Problem for CX-focused generative AI. - guest blog by Jonas Berggren.

What happened?

Read more

Leading Voices: most brands fail at agentic AI. Our solutions experts share how yours can win.

Your RFP is asking yesterday’s questions about tomorrow’s business.

What KPIs should you track in customer service? The data you need - and how to use it.