Why Did My AI Say That? Fixing a Wrong Answer

When your AI assistant gives a customer a wrong answer, the cause is usually one of two things: a rule in its instructions, or a piece of business info it read. Ask Maya "why did my AI say that?" and she reads the conversation, finds which one it was, and proposes a fix you can test on sample messages before any customer sees it. In a Gartner survey of 3,566 customers run in early 2026, only 27% said they would try a chatbot again after a bad experience.

What does Maya look at when you ask why the AI said something?

More than the inbox shows. Your AI employees keep a record of how each reply was built, and Maya reads it alongside the messages (the Maya page has the short version):

She finds the conversation from a customer name, a channel or a rough time: "Ahmed, on Instagram, this week".
She reads the trace behind each AI reply: which part of your assistant answered, what it looked up, what came back, which language it detected, and whether the final quality check flagged anything.
She names the cause, down to the exact rule or piece of info.
She proposes a fix as a card with the before and after. By default, nothing changes until you approve it.

Where do wrong answers usually come from?

Most bad replies fit a handful of patterns.

What the customer saw	Usual cause	What Maya suggests
An old price	A price typed into the assistant's rules that no longer matches your price list	Ask which price is right, then remove the typed price so the AI reads your price list, or correct it
A promise you don't make	A rule worded too broadly, or two rules that contradict	Narrow the rule, or drop one of the pair
An answer when it should have passed the chat to you	No handover rule for that topic	Add the topic to your handover rules
"I'm not sure" to a simple question	The info isn't in your business details	Add an FAQ, a service or the missing detail

The first row is easy to miss: you update the price list and forget the price typed into the rules. Maya can scan your assistant's instructions for exactly that, flagging prices, hours and contact details written in by hand where they should come from your business data. Our guide to keeping prices, services and FAQs current by chat covers the other half.

Two sources of truth that disagreed cost Air Canada in Moffatt v. Air Canada (2024). Its chatbot told a grieving customer they could claim a bereavement fare after travelling; the airline's own bereavement page said they couldn't. The tribunal held the airline responsible for everything on its website, chatbot included, and ordered it to pay CA$812.02, of which CA$650.88 was damages.

What does fixing it look like in a chat?

An example (the name and prices are made up):

You: Why did the AI quote the old price to Ahmed? Whitening is $180 now.

Maya: I found his Instagram chat from Tuesday. Your price list says $180, but your assistant's rules still say "teeth whitening costs $150". I can remove that line so it always reads the price list, or change it to $180. Which would you like? I'll test it first.

You: Remove it, and test it.

Maya: I ran "how much is teeth whitening?" through your assistant with the change. It now says $180. Here's the change for your OK.

The card shows the line being removed. Approve it, reject it, or tell Maya what to adjust. More on rule changes in how to change your AI assistant's replies by asking.

How do you test a fix before customers see it?

Test runs you ask for. Say "Test it: a customer asks for a refund after 30 days" and Maya runs that message through your live assistant and shows you the reply. Nothing is sent, nothing is booked, and the test run itself isn't billed. You can test a short back-and-forth, or ask "what if we changed this rule?" before any change exists.

Verify changes, a setting that's on by default. When Maya shows you a change card, she tests it first: she replays a test message with the change applied and checks the reply does what you asked. The card says Testing… and can't be approved until the test finishes. Where she can, she also replays a few of your real recent customer messages, to catch a fix that spoils replies that were fine. A fix for a disliked reply is tested on the real message that went wrong. This uses training credits; you can switch it off in Maya's settings, though we'd leave it on.

Rule edits get one more check: Maya looks through your whole set of instructions for duplicate or contradictory rules before the card reaches you. And in "just do it" mode (Bypass in the app), a change that fails its test still waits for you.

Can you start from a reply you disliked?

Yes, and it's often the quickest route. Give the reply a thumbs-down in the inbox and write a line about what was wrong. It then appears in the Feedback list on your AI employee's page, with a Resolve with AI button (still in beta). Press it and Maya opens a chat that already holds the conversation and your note, then traces the reply and proposes a tested fix. Handy when a teammate reads the inbox: they flag it, you sort it out later.

What happens to the customer who got the wrong answer?

Maya fixes the assistant, not the past conversation. She never messages customers herself, so if Ahmed needs a correction, you or your team reply from the inbox, the same rule as when Maya books, moves or cancels appointments.

Changes to how your assistant replies are saved as versions. If a fix turns out worse than the problem, ask Maya to go back to the earlier one.

What can't Maya do here?

Message the affected customer. That stays with you and your team.
Test a phone call. Test runs are typed messages. She can read call transcripts but can't simulate a call.
Promise a perfect result. A test covers a few messages and AI replies vary, so watch the next real chats on that topic.
Invent missing information. If the right answer isn't in your business details, she'll ask you for it.
Run the automatic test with no training credits left. It's skipped, and the card says so.

For the method underneath, see our guides on writing AI agent instructions and stopping AI agents from hallucinating.

Got a reply you'd like explained? Text Maya on WhatsApp with the customer's name and what went wrong.

FAQ

Does testing a fix send anything to my customers?

No. Test runs use a made-up conversation, so nothing reaches a customer and nothing is booked or ordered.

Does it cost credits to ask Maya why the AI said something?

Chatting with Maya uses plan credits as usual. Test runs you ask for aren't billed on top; the automatic Verify changes check uses training credits.

Who on my team can do this?

The owner, admins, and team members who have been given the Maya privilege.

Can I do this from WhatsApp?

Yes. Text Maya the customer's name and what was wrong. A fix whose test failed comes as a Review link to the app instead of a one-tap Approve.

Sources: Gartner, "Only 27% of Customers Would Try a Chatbot Again After a Negative Experience" (September 2026); Moffatt v. Air Canada, 2024 BCCRT 149.

Why Did My AI Say That? Finding and Fixing a Wrong Answer