When AI Gets It Wrong: The Real Stories Behind Artificial Intelligence’s Biggest Mistakes

By Awoyode David

On a quiet afternoon, a lawyer walked into a courtroom carrying what seemed like a well-prepared legal brief. Every citation looked professional. Every case appeared legitimate. There was just one problem: several of the court cases never existed. They had been confidently invented by ChatGPT. Within days, the incident made international headlines, leaving a judge questioning not just one attorney’s judgment, but the growing trust society places in artificial intelligence.

It was a wake-up call. As AI becomes embedded in our workplaces, schools, hospitals, and daily lives, understanding its limitations is no longer optional—it is essential.

AI Is Everywhere—And So Are Its Mistakes

Artificial intelligence has experienced explosive adoption in recent years. According to McKinsey’s 2025 Global Survey, 78% of organizations now use AI in at least one business function, a significant increase from previous years.  

Meanwhile, ChatGPT reached hundreds of millions of weekly users within two years of launch, making it one of the fastest-growing consumer technologies ever created. With so many people relying on AI to answer questions, summarize information, write code, and even provide legal or medical guidance, the consequences of inaccurate responses have become increasingly serious.  

The Lawyer Who Submitted Fake Court Cases

Perhaps the most famous AI failure occurred in 2023 when New York attorneys used ChatGPT to help prepare a legal filing.

The chatbot generated multiple court precedents complete with case names, quotations, and legal reasoning. The problem? Several of the cases were entirely fictional.

The judge described the incident as unprecedented, and the lawyers were sanctioned after admitting they had failed to verify the AI-generated citations.

The lesson wasn’t that AI is useless—it was that AI can produce convincing misinformation with remarkable confidence.

Google’s AI Search Told People to Eat Rocks

In 2024, Google’s AI Overviews feature became the subject of global criticism after users discovered bizarre responses appearing at the top of search results.

Among the most viral examples were recommendations suggesting that people eat small rocks for minerals or use glue to keep cheese on pizza.

Google later explained that many of these answers originated from satirical or unreliable online sources and announced improvements to reduce such errors.

The incident demonstrated that even technology from one of the world’s largest companies is not immune to AI hallucinations.

Air Canada’s Chatbot Cost the Company in Court

Artificial intelligence doesn’t only make factual mistakes—it can also create expensive business problems.

In Canada, an Air Canada customer asked the airline’s chatbot about bereavement fares after purchasing a ticket.

The chatbot incorrectly stated that the customer could apply for a refund afterward.

When the airline later refused to honor that information, the dispute reached a tribunal.

The court ruled that Air Canada was responsible for the chatbot’s incorrect advice and ordered compensation, rejecting the airline’s argument that the chatbot was a separate legal entity. The decision reinforced an important principle: companies remain accountable for what their AI systems tell customers.  

AI Can Sound Certain Even When It’s Wrong

One of the biggest dangers isn’t simply that AI makes mistakes—it’s how confidently those mistakes are delivered.

Researchers at Carnegie Mellon University and Microsoft Research found that users assisted by AI were less likely to recognize uncertainty and more likely to accept incorrect answers simply because they were presented confidently. This phenomenon, known as automation bias, can reduce critical thinking when people rely too heavily on AI-generated information.  

Hallucinations Are Still a Major Challenge

Researchers refer to fabricated AI responses as “hallucinations”—instances where a model generates information that appears credible but is false.

A 2025 Nature feature reported that hallucinations remain one of the most difficult problems facing modern large language models, despite rapid advances in AI capabilities. Experts noted that current techniques can reduce hallucinations but cannot eliminate them entirely.  

Similarly, a 2025 study published in Scientific Reports analyzed thousands of user reviews of AI-powered mobile apps and found that hallucinations continue to be one of the most common complaints among users, particularly when AI invents facts, references, or explanations that sound believable.  

Awareness Matters More Than Fear

None of these incidents mean people should stop using artificial intelligence.

AI is already transforming healthcare, education, scientific research, software development, and countless other industries. Used responsibly, it can save time, improve productivity, and unlock new opportunities.

However, awareness is the difference between using AI as a powerful assistant and treating it as an unquestionable authority.

The safest approach is simple:

• Verify important facts using trusted sources.
• Never rely solely on AI for legal, medical, or financial decisions.
• Treat AI-generated citations and statistics as information that requires independent verification.
• Remember that fluent language is not the same as factual accuracy.

Conclusion

Artificial intelligence is one of the most powerful technologies of our generation, but intelligence without verification can become misinformation at unprecedented speed. The stories of lawyers submitting imaginary court cases, companies paying for chatbot mistakes, and search engines producing absurd advice remind us that AI is neither perfect nor infallible. As these systems continue to shape how we work, learn, and make decisions, perhaps the most valuable skill of the AI era will not be knowing how to use artificial intelligence—but knowing when not to trust it, because if machines can sound right while being wrong, are we asking enough questions before believing the answers?