Test Scenarios

Test Scenarios let you check that your assistant answers the way you expect — before your visitors do. You write the questions that matter, say what a good answer looks like, then run the scenario to get a pass rate and a detailed report. It's the safest way to make a change (new instructions, new knowledge) without quietly breaking something else — and it even suggests the documents to add when an answer falls short.

Where to find it

1. Open Administration → My Assistants → [your assistant].
2. Open Test Scenarios.

The page lists your scenarios with their Pass Rate and Last Run, and shows which are Active or Inactive.

---

Create a scenario

Click Create Scenario and fill in the Basic Information:

• Scenario Name — a short label (for example, "Product Knowledge Test").
• Description — optional notes on what this scenario checks.
• Active — whether the scenario is included when you run your tests.

Then add Questions. For each one (Add Question):

• The question a visitor might ask.
• Expected Answer (optional) — what a correct answer should contain.
• Expected Business Result — the outcome that really matters for your business (for example: "The visitor is told the 14-day refund window and offered a return."). This is used for business validation.

Import from FAQ

Already built an FAQ? Use Import from FAQ to turn your existing questions into test questions in one click, then add the expected results.

Write scenarios for the questions that matter most — your top support topics, anything with a precise answer (prices, policies, hours), and any bug you've fixed so it never comes back.

---

Run scenarios

• Run / Execute — run a single scenario, for example while you're editing it.
• Run your scenarios after every significant change to your assistant's instructions or knowledge base.

While a test is running you'll see its Progress; when it's done you can View Results.

---

Read the results

Each run produces an Overall Score and marks each question as Passed or Failed, with the test Duration. Open Test Results for the Detailed Results, where each question shows:

• The AI Response the assistant actually gave.
• Documents Used — the knowledge the assistant drew on.
• Missing Documents — knowledge that should have been there but wasn't.
• A Business Justification of whether the expected business result was met.

At the top, Insights & Recommendations summarize what's working and what to improve.

---

Apply corrections

When a question fails because knowledge is missing, Test Scenarios can suggest fixes for you:

• Suggested Documents to Add — ready-made knowledge entries that would have produced the right answer.
• Apply Corrections (or Batch Corrections to handle several at once) — add the suggested documents to your knowledge base in one click.
• Rerun Test to confirm the scenario now passes.

This turns a failing test into an improved assistant in a couple of clicks.

---

Execution history

Every run is kept in the Execution History, with its pass/fail counts and duration — so you can see how your assistant's results evolve over time and confirm that a green run really means nothing broke.