LLM Chatbot Testing Checklist

What should you set up before testing?

Agree what the chatbot is for, what it must never do and what a correct answer looks like. Then build a golden set: real questions with expected answers that the business owner has approved. Everything else in this checklist is measured against that set, so do it first and keep it under version control.

• Purpose, audience and out-of-scope topics written down
• Golden questions taken from real support history
• Approved answers, with the source document for each
• A test environment with test accounts
• Versions recorded: model, prompt, settings and documents

Is it accurate?

Run every golden question several times and compare each answer with the approved one. Check the facts it must contain rather than exact wording. Also ask questions whose answer isn’t in your documents, and check that the chatbot says it doesn’t know instead of inventing something.

• Required facts are present
• No invented facts, names, links or policies
• An honest "I don’t know" for unanswerable questions
• The same answer quality across repeated runs
• Correct handling of questions that look alike but have different answers

Does it stay inside its boundaries?

Test what the chatbot refuses to do as carefully as what it does. Ask for off-topic help, advice it shouldn’t give, and actions it can’t take. It should decline politely, stay in its role and point to a person or another channel when it can’t help.

• Off-topic requests
• Medical, legal or financial advice, if that is out of scope
• Requests for actions it has no tool for
• A handoff to a human that works and passes the context along

Can it leak anything it shouldn’t?

Try to make the chatbot reveal its instructions, other users' data, internal notes or personal information. Test both direct requests and indirect ones, such as text hidden in a document or web page it reads. Scan every reply with code for personal data and internal terms.

• Attempts to extract the system prompt
• Requests for another user's data
• Internal URLs and notes in replies
• Personal data patterns such as email addresses and phone numbers

For a deeper, attack-focused test, see prompt injection testing.

How does it behave in a long conversation?

Many failures only appear after several turns. Test follow-up questions, changes of mind, corrections from the user and very long conversations. The chatbot should remember what matters, stay consistent with its earlier answers and not drift away from its instructions as the conversation grows.

• Follow-ups that depend on earlier turns
• A user who corrects the chatbot
• Switching between languages
• Restarting the conversation

What operational limits should you check?

Measure how long answers take, what each answer costs and what happens when things go wrong. Test slow or failing model calls, rate limits, empty input, very long input and unusual characters. The chatbot should fail politely and visibly, never silently or with a half-written answer.

How do you keep testing after release?

Turn the golden set into a regression suite that runs after every change to the prompt, model, settings or documents. Add each real failure you find in production as a new test. Review a sample of live conversations regularly, because customers will ask things your test set didn’t.

If you would like a second pair of eyes, read about the chatbot audit or request a quote.