Agentic Testing
An AI agent can run a Manual Test Execution for you, not just plan one. It reads each case's steps as data, exercises them against your app with its own tooling, and writes pass or fail back step by step - into the same execution, with the same evidence a human tester would have left.
Why this exists
Manual regression is where release time goes. Automation covers the stable paths, and everything else - a new feature, a fiddly flow, a one-off environment - falls back to a person clicking through a checklist.
AI agents can already do that clicking. What they could not do is leave a trace anyone can act on: the run happens in a chat window, the result is a paragraph of prose, and nothing reaches the system that decides whether the release ships.
Agentic testing closes that gap. M00N Report exposes the manual run itself as something an agent can operate - addressable cases, steps as structured data, a write-back for results - so the agent's work lands as execution evidence instead of a transcript.
What you get out of it
Per-step pass/fail, the failing step named, an issue note on it, duration and notes.
The agent acts as its key's account, under that role, and every call lands in the MCP audit log.
The agent reports facts about steps. Case and group status follow on their own.
Reporting is atomic and idempotent, so an interrupted or repeated run converges.
Both write to the same case rows, so either can take over mid-run.
No built-in AI, and nothing is sent to a separate AI service. The doing is your agent's.
Because the results land in a real execution, everything downstream keeps working: release readiness counts them, coverage sees them, and the execution links to a release or a Jira ticket like any other.
Prerequisites
- An MCP client connected to M00N Report - see the Quick Start or the Remote Connector.
- An MCP key whose account has Executions: read and write in the project. A key can never exceed the permissions of the user it belongs to.
- A way for the agent to actually exercise your app: a browser MCP such as
@playwright/mcp, an HTTP client, a terminal - whatever the flow needs. - An execution containing cases with steps. A case with no steps gives the agent nothing to follow.
Run one, step by step
Build the execution
Assemble it in the app under Executions, or let the assistant do it: create_execution, then add_suites_to_execution, add_collections_to_execution or add_cases_to_execution to populate it.
Cases are snapshotted into the execution when they are added, so editing a case later does not rewrite a run already in flight.
Give the agent hands
Add whatever tool the agent needs to drive your app alongside the M00N server. A browser MCP is the usual choice:
{
"mcpServers": {
"m00n": {
"command": "npx",
"args": ["-y", "@m00nsolutions/mcp-server"],
"env": {
"M00N_API_URL": "https://m00nreport.com",
"M00N_API_KEY": "m00n_mcp_your_key_here"
}
},
"playwright": {
"command": "npx",
"args": ["-y", "@playwright/mcp@latest"]
}
}
}Start the run
Use the run_manual_execution prompt. It lists the project's runnable executions and hands the agent the whole procedure. If your client has no prompt picker, ask in words:
"Run the Sprint 48 Smoke execution against staging: follow each case's steps in the browser, record pass/fail per step, mark blocked cases with the reason, link failures to Jira, and set the final execution status."What the agent does
get_project- read the project's status vocabulary. Statuses are configurable per project, so nothing is hardcoded.update_executionwithin_testing- mark the run started.get_execution_cases- every case row with itsexecutionCaseIdand its steps: index, action, expected result, current status.- Exercise the steps against the system under test with its own tooling.
report_case_resultper case - the outcome, step by step.update_executionwith the terminal status when the run is done.
A single report looks like this:
{
"executionCaseId": "9f1c8e04-...-c4e2",
"status": "auto",
"steps": [
{ "index": 0, "status": "passed" },
{ "index": 1, "status": "passed" },
{
"index": 2,
"status": "failed",
"issueDescription": "Checkout returned 500 after applying promo code SUMMER25"
}
],
"causedBy": "Step 3 - apply promo code",
"notes": "Reproduced twice on staging, Chromium 141",
"durationMinutes": 4
}"auto" derives the case status from the steps just written, so the agent never has to reason about precedence. The call returns the resulting case status, the group status and fresh execution progress.
Watch it land
Open the execution while the agent works. Cases flip status as reports arrive, failing steps carry their issue notes, and progress counters move. Everything the agent writes reads back through get_execution_cases too, so it can track its own completion.
Review, then decide
Read the failures, not the whole run. Ask for the shortlist, re-check anything surprising yourself, and link real defects out with add_external_link to Jira, Linear, GitHub or GitLab.
"Summarise that run: which cases failed, on which step, and what the issue note says. Create a Jira bug for each distinct failure and link it to the case."The tools it uses
| Tool | Type | Role in the run |
|---|---|---|
get_project | Read | The project's configurable status values |
list_executions | Read | Find the execution to run |
get_execution_cases | Read | Case rows and their steps - the procedure to follow |
report_case_result | Write | Per-step and per-case results, notes, duration |
update_execution | Write | Start the run, then close it with a terminal status |
set_execution_source_status | Write | Manual override of a group status, rarely needed |
add_external_link | Write | Link a failure to a Jira, Linear, GitHub or GitLab ticket |
Full parameter reference on the Available Tools page.
How statuses roll up
The agent reports steps. Everything above a step computes itself, which is what keeps an agent-run execution consistent with a human-run one.
| Level | How it is set |
|---|---|
| Step | Reported directly: not executed, passed, failed, blocked or skipped, with an optional issue description |
| Case | "auto" derives it from the steps in precedence order - failed, then blocked, then in testing, then passed. An explicit status wins if one is passed |
| Source group | Rolls up automatically: nothing executed is not started, some executed is in progress, all executed is completed - or blocked when a finished group still holds a blocked case |
| Execution | Set explicitly at the end with update_execution |
get_project before it reports anything.Where it fits best
Two workloads carry most of the value: regression testing, where the same scripted pass runs every release, and exploratory testing, where nothing is scripted yet and the point is to find out what breaks.
The strongest fit. The agent follows written steps rather than habit, so the pass runs the same way every time and never loses attention on case 60. Hand it the regression execution overnight and start the morning on the failures.
Give it a charter instead of a checklist and it probes the app like a tester would. The difference is what survives: findings become real cases via create_test_case and defects via add_external_link, so exploration turns into regression coverage instead of evaporating in a chat log.
Same execution, different target. Cheap enough to run against a preview deployment, and the result is comparable run to run.
Run a freshly written case with an agent first. If the steps are too vague for an agent to follow, they were too vague for a new tester too.
Split the execution: the mechanical cases to the agent, the judgement calls to a tester. Both write to the same run.
Limits worth knowing
- The agent brings the doing. M00N Report has no built-in AI and does not drive your app. Without a browser or API tool wired into the same client, the agent can read the run and nothing more.
- No attachments on a manual case run over MCP. Screenshots and traces attach to automated launch results, not to agent-reported manual cases. Put the evidence in
issueDescriptionornotes, or link it out withadd_external_link. - Results are only as good as your steps. "Check the page works" gives an agent nothing to verify. Cases with concrete actions and expected results are what make the run repeatable.
- Review the failures. An agent can misread a UI exactly like a tired human. Treat a failed case as a report to confirm, not a verdict.
- Permissions are real. If the key's account cannot write executions in that project, the run stops at reading. Check the account's role first.