Simulations

Simulations run scripted conversations against your agent and check whether the agent behaves as expected. Use them to catch problems before visitors encounter them. Open the Testing tab in the Navattic Agent editor to access your simulation suite.

​

Creating a simulation

Click Add simulation to create a new simulation. Each simulation defines:

• Persona: The visitor the simulation plays. Give the persona a name, role, and behavior style (for example, Cautious, Eager, or Skeptical).
• Expected outcome: The conversation outcome the agent should reach by the end of the session.
• Max turns: The number of conversation exchanges before the simulation ends.
• Questions the buyer asks: Specific questions the persona asks during the session.
• What the buyer does: Instructions for how the persona engages (for example, objections they raise or flows they request).
• Criteria: Specific behaviors or phrases you want to verify. The simulation checks each criterion when grading the run.

Navattic can also generate a starter suite for your agent. Click Generate to create a set of simulations based on your agent’s stages and outcomes.

​

Running simulations

Click Run all to start the suite. Navattic runs every simulation in sequence and grades each one against the expected outcome and criteria. A pass rate bar at the top of the sidebar updates as runs complete. Simulations that you edited since the last run are marked Queued. Re-run the suite after editing your agent to get an updated pass rate.

​

Reading results

The sidebar groups your simulations by status. After a run, each simulation shows one of the following statuses:

• Excellent: Every run reached the expected outcome and met all criteria.
• On track: The agent reached the expected outcome, but a run missed a criterion or some runs did not reach it.
• Failing: The agent did not reach the expected outcome often enough, or fired the wrong outcome.
• Out of date: An outcome the simulation tests was edited or deleted. See Out-of-date simulations.
• Queued: The simulation has not run yet, or was edited since the last run.

The sidebar shows these under four group headings: Needs review (Failing and Out of date), Queued, On track, and Excellent. Click any simulation to open its detail view. Where a run needs attention, a recommendation card shows what went wrong and, where possible, proposes a specific change you can apply directly from the card.

​

Out-of-date simulations

When you edit or delete an outcome that a simulation tests, Navattic marks the simulation Out of date. Out-of-date simulations are left out of runs and the pass rate until you resolve them. Open the simulation and choose one of the following:

• Regenerate: Rewrite a generated simulation to test the current outcome. Click Regenerate all in the group heading to regenerate every generated simulation in the group.
• Keep as is: Keep the simulation unchanged because it still tests the right thing.
• Edit simulation: Update a simulation you wrote yourself, or one that cannot be regenerated.
• Delete: Remove a simulation whose outcome was deleted.

​

Marking a run as passing

Sometimes a run reaches the expected outcome but misses a criterion that does not matter. To count that run as passing:

1. Open the simulation, then click Mark as passing on the recommendation card. You can also open the run result at the bottom of the detail view and click Mark as passing there.
2. Choose why the run should pass: The grader misread the conversation, The criterion is wrong, The outcome handles it, or The buyer never gave the agent the chance.
3. Optionally add a comment, then click Mark as passing.

Marking a run as passing clears its notes, so a simulation with no other notes moves to Excellent. Click Undo on the card to reverse it.

​

Applying a recommendation

A recommendation may include a proposed fix: a concrete text change to a stage goal, topic message, outcome condition, outcome final message, or brain entry. To apply a recommendation:

1. Review the proposed change shown on the card. The card displays the current live text so you can compare it with the suggestion.
2. Click Apply recommendation to patch your agent with the proposed change. Navattic checks that the text has not changed since the suggestion was generated. If it has, the apply is refused and you can review the current version first.
3. After applying, the simulation is marked Queued. The pass rate does not change until you re-run the suite.

To reverse an applied recommendation, click Undo on the same card. Navattic restores the exact text that was in place before you applied the change.

​

Edit or regenerate a recommendation

Click the More recommendation actions menu on the card to adjust a recommendation before you apply it:

• Edit recommendation: Change the suggested text, or pick a different part of your agent to update, then click Apply recommendation. The card then reads Edited recommendation applied.
• Regenerate: Ask Navattic for a new recommendation for the same run.

​

Pass rate

The pass rate bar shows what percentage of your simulations passed in the most recent run. Applying a finding does not update the pass rate. Re-run the suite after making changes to earn credit for each fix.

​

Read results via MCP

Connect the Navattic MCP server with the Navattic Agents scope to retrieve simulation transcripts and scores from outside the app. Use this to grade runs with your own LLM judge or pull results into a CI pipeline. List the simulation runs for an agent to get run IDs, then retrieve a run’s details to see the conversation IDs for each simulated session. From a conversation ID, you can fetch the full transcript and how Navattic scored it: the outcome that fired, per-criterion verdicts and rationale, and any open findings or errors.