> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hubtalk.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Testing an agent

> Checking an agent with scenarios: test conversations before publishing, a judge, a launch-ready result and what to fix.

# Testing an agent

The **Test** button in the editor toolbar opens the test panel. It has two tabs:

* **Voice** / **Chat** — you talk to the agent yourself. The tab is named after the open mode: **Test** opens a call — the **Voice** tab — and its menu switches to **Chat** and back; see [Test before launch](https://docs.hubtalk.ai/v4/platform/agents/building.md#test-before-launch).
* **Scenarios** — the platform talks to the agent for you, following scenarios you describe in advance, and tells you whether the agent is ready to launch. This page is about this tab.

A check is a batch of test conversations with the agent. Test callers talk to it following the scenarios. After each conversation a judge decides whether the agent did what was expected, and at the end you get a result: whether the agent is ready to launch, and what to improve.

The check makes no real calls and writes nothing to **History** — the conversations stay in the check result.

## How it works

1. **Scenarios.** You describe who calls and what you expect from the agent, or let the platform generate scenarios from the agent settings.
2. **Check.** Test callers talk to the agent following every enabled scenario, in parallel.
3. **Judge.** After each conversation a judge model compares it with the expectation and marks the line where the agent went wrong.
4. **Result.** **Ready to launch**, **Not ready** or **Not everything was checked** — by the thresholds of the organization.
5. **What to improve.** The platform groups the failures and suggests changes to the agent.
6. **Fix with Copilot.** Copilot proposes a change, you apply it, and the scenario is checked again.
7. **Publishing.** The publish dialog shows whether the check passed.

## Scenario

A scenario describes one conversation. It is stored with the agent and outlives the checks: the same set can be run after every change.

| Field | What it is |
| - | - |
| **Title** | A short name of the situation: "Not the contract owner calls" |
| **Caller** | Who calls and how they behave — 2–3 sentences |
| **Knows about themselves** | What the caller can say about themselves: name, contract number, city |
| **Agent variables** | Values of the agent variables in this conversation. The block is folded; it holds the agent's variables and your own "name + value" rows |
| **Manner** | One of seven — below |
| **Expected from the agent** | What the agent must do. The judge decides by this field |
| **Break-in attempt** | An attack scenario: the caller tries to make the agent break its rules |
| **With the agent's knowledge base** | Ticked by default. Untick it to check how the agent answers in this scenario without its knowledge bases. Disabled for an agent without knowledge bases |

**Manners:** Ordinary; In a hurry, irritated; Rude, threatening; Confused; Elderly, asks again; Answers in one word; Slang, typos.

Marks of a scenario in the list: **Attack** — an attack scenario, **Own** — added by hand, **No knowledge base** — "With the agent's knowledge base" is unticked. An attack is a scenario like any other: you can turn it on and off, edit and delete it, and have as many of your own as you like. The checkbox turns a scenario on and off: a disabled scenario stays with the agent but does not run.

Enabled scenarios come first in the list, disabled ones last. The first 10 rows are shown; **Show more (N)** opens the next ones. **Clear selection** in the list header unticks every scenario; when none is ticked, it turns into **Select all**.

## Where scenarios come from

* **Your own** — the **Add your own scenario** button.
* **Generation** — **Generate from the agent settings**. Choose how many with **− 8 +**, from 3 to 20. With **with the knowledge base** ticked, callers will ask about facts from the agent's knowledge bases. Scenarios appear in the list as they are ready; **Cancel** stops the generation and keeps what was already created. **More** adds scenarios without repeating the existing ones and asks for the number and the knowledge base the same way. If the agent has no attack yet, one of the ordered scenarios will be an attack: order 3 and you get 3, one of them marked "Attack". Scenarios are written in your cabinet language; lines the caller must say word for word stay in the agent's language.

**Scenarios in another language.** A scenario written in a cabinet with another language is shown to you translated into your cabinet language — in the list and in the form. The model translates it once, and the translation is kept for everyone with that cabinet language; quoted words, names, numbers and variables are not translated. The form marks such text **Auto-translated from …**, and **Show original** shows the source text. The check and the judge always work on the original, so a translation cannot change the result. If you edit a scenario's text and save it, your version becomes the new original and the old translations are dropped.

* **Copilot** — ask in the chat: "add a scenario where the caller…". Copilot saves it with the same fields.
* **From coverage** — after a check: **Create scenarios for these nodes** (see [below](#what-else-the-result-shows)).

If the agent has no enabled scenario, the check runs five base callers: calm, in a hurry, confused, off-topic and attacking. Base callers are not saved with the agent. If there are enabled scenarios, exactly those run — nothing is added beyond what is ticked.

**Delete all scenarios** in the list header removes the agent's check scenarios. Past check results stay.

## The check

The **Check the agent** button in the tab footer shows how many scenarios will run — the number of ticked rows. With every row ticked it matches the number in the list header.

* **What is checked** — the draft or a published version. The choice is the chat test's: **What to test** in the test configuration on the **Chat** tab (the summary button, for example **Draft · rigid · mock**). A draft with schema errors cannot be checked: the button stays disabled until they are fixed.
* **A conversation** — up to six turns; the agent speaks first. Conversations run in parallel, and the progress bar shows "Conversations in progress · N of M". **Stop** interrupts the check: no new conversations start, and the ones in progress finish their current turn — meanwhile the tab shows **Stopping the check** and how many conversations are still running. Scenarios that never started get **Not run**. The finished conversations of a stopped check open from its result, with **← History** above it (or **← To scenarios** if there are no past checks).
* **Agent functions** answer with test stubs: the check calls nothing outside. The judge knows this and judges the logic of the step, not specific values. Functions without a test answer are listed before the start.
* **The agent model is unavailable** — the check does not start, and the tab says so: "The agent model is unavailable: … Check the LLM settings of the agent". If the model fails in the middle of a check, the check stops with the same reason instead of marking every scenario separately.
* **One check at a time** per agent: while one runs, a second cannot start.

### How the judge decides

The judge reads the whole conversation once, at the end. The scenario's expectation and the agent's rules are its criteria, and it decides for each one whether it was met. A scenario passes when no criterion is broken. In the conversation card the judge explains the decision in the cabinet language and marks the line where the agent went wrong — **Error here**.

If the judge skipped a criterion, it is asked once more. If it still does not rate it, the criterion does not count as a failure, and the conversation card shows **The judge did not rate: …**.

If the judge could not answer or the conversation did not happen, the scenario gets **could not run**. This is not an agent error: such a scenario does not count as a failure; repeat the check.

## The result

| Result | When |
| - | - |
| **Ready to launch** | At least the threshold of ordinary scenarios passed, and breaches are within the allowed number |
| **Not ready** | The threshold is not reached, or the agent was broken into |
| **Not everything was checked** | Some scenarios could not run, or there was no judged scenario of some kind |

The result line reads like this: "Not ready · 3 of 5" — 3 conversations of 5 failed; with **Ready to launch** the number counts the passed ones. Attacks are counted along with the other scenarios. Expand the line to see the thresholds and the number of breaches.

**Thresholds** are set per organization: at least 95% of scenarios must pass, with 0 breaches.

**A spot check** — **Check again** on one scenario, or a run from Copilot. It updates that scenario's outcome but gives no "ready to launch" result: that needs a full check.

**The result is outdated** — the agent changed after the check: the draft was edited or a new version was published. Check again before publishing.

### What else the result shows

* **The Scenarios and What to improve tabs.** Failed scenarios come first, then those not run, then other outcomes, and the passed ones last.
* **Knowledge base** — a mark on a scenario where the agent answered from the knowledge base; in the conversation card such a line is labelled **Answered from the knowledge base**.
* **Comparison marks** against the last full check: **Broke** — passed before, fails now; **Fixed** — the other way round; **New**; **not checked**; **could not run**.
* **Node coverage:** "The conversations did not reach N nodes of M" and the first names of those nodes. The check does not know what the agent says in nodes it never reached. **Create scenarios for these nodes** adds scenarios that lead there.
* **The conversation card:** the whole dialogue, the judge's verdict, "Expected from the agent", the variables of this conversation and the **Error here** mark. The node with the error is highlighted on the canvas.

## What to improve

After a full check with failures, the platform groups the errors and, for each group, writes what is wrong, why, in which scenarios, how to fix it and where — the general prompt, a specific node or the settings. Recommendations are prepared separately from the result: the result is shown at once, with "Preparing recommendations…" next to it. If they did not make it in time, **Get recommendations** asks again.

## Fix with Copilot

**Fix with Copilot** on a failure or on a recommendation group opens [Copilot](https://docs.hubtalk.ai/v4/platform/copilot.md) with the conversation from the check attached and a ready question. Copilot proposes a change to the agent; a person applies it with **Apply**. Then "The change was applied to the draft" and **Check the scenario again** appear. After the fix the conversation card shows **What changed in the agent** and **Before → After**.

## Publishing

The publish dialog shows two rows:

* **Schema** — schema errors block publishing.
* **Check** — passed; failed (with **Open the results**); not run (with **Check the agent**); the agent changed after the check (with **Check again**).

The check does not block publishing: if it failed, the button becomes **Publish anyway**.

## Check history

If the agent has been checked before, the tab opens on the scenario set with the **Past checks** block above it: date, version, result and number of scenarios. The last three are shown, the rest under **Show all (N)**. A row opens its result; **← History** returns to the list.

## Who can run checks

The **Agents and editor** section is open to the administrator and the developer, so the check is theirs only (see [Roles](https://docs.hubtalk.ai/v4/platform/roles.md)).

| Action | Administrator | Developer | Manager | Member |
| - | - | - | - | - |
| See the Scenarios tab and the results | ● | ● | — | — |
| Add, generate and delete scenarios | ● | ● | — | — |
| Start and stop a check | ● | ● | — | — |
| Apply Copilot's changes | ● | ● | — | — |

## Good to know

* Check conversations are test conversations: they do not enter **History** or call analytics.
* The check runs with test answers of functions: the agent's real integrations are not called.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.