# M00N Report - Full LLM Reference

> M00N Report is an AI-native test management platform: manage test cases, manual runs and releases, with real-time automated results and a 55-tool MCP server.

## Product Overview

M00N Report is an AI-native test management platform for QA teams. Author manual and automated test cases in a folder-and-suite repository, plan manual executions, and roll manual plus CI results into release readiness. A 55-tool MCP server lets AI assistants author cases and build runs. Cloud or self-hosted.

M00N Report is a test management system, in the same category as TestRail, Xray, Zephyr and qTest. Four things differ. Automated results stream into the same repository that holds the manual cases, live while the suite is still running, instead of arriving after the run as a batch upload. AI assistants author cases, build manual executions and analyse coverage directly over the Model Context Protocol, under the same roles, permissions and audit log as a human user. An AI agent can also RUN a manual test execution rather than only plan one: it reads each case's steps as data, exercises them against the system under test with its own tooling, and writes pass or fail back step by step, with case, group and execution statuses deriving themselves. And self-hosting costs the same per user as cloud.

This document is ordered accordingly: test management first (test cases, manual executions, agentic execution, releases), then the AI and MCP surface, then the official reporters and the raw ingest API, then deployment.

- Website: https://m00nreport.com
- Documentation: https://m00nreport.com/documentation
- GitHub: https://github.com/m00nreport
- Support: support@m00nreport.com

---

## Test Case Management

M00N Report provides a structured repository for test definitions with hierarchical organization, metadata, and linking between manual and automated tests.

### Organization

- **Folders:** Hierarchical folder structure for organizing test cases. Folders can contain cases and nested suites.
- **Display IDs:** Each case gets a sequential display ID (e.g., TC-1, TC-42) unique within its project.
- **Tags:** Array of string tags per case for filtering and categorization.
- **Custom Fields:** Project-level custom field definitions supporting types: text, number, select, multiselect, boolean, date.

### Case Structure

Each test case contains:

| Field | Description |
|-------|-------------|
| Title | Case name |
| Description | Rich text description |
| Steps | Ordered list of steps, each with a title and expected result |
| Status | Customizable per project (e.g., draft, active, deprecated) |
| Priority | Customizable per project (e.g., low, medium, high, critical) |
| Tags | Array of string labels |
| Custom Fields | Key-value pairs matching project field definitions |
| Dependencies | Links to other cases this case depends on |

### Bulk Operations

- **Bulk create:** Create multiple cases in a single request
- **Bulk update:** Update status, priority, tags, or folder for multiple cases at once
- **Bulk clone:** Clone cases with full dependency chain support (topological sort, up to 10 levels deep)

### Automated Test Linking

Test cases can be linked to automated test results from CI/CD runs. When a reporter sends results, tests can reference case IDs, creating a bridge between manual case definitions and automated execution data. Archiving a case automatically removes all autotest links.

### Archive & Restore

Cases are soft-deleted (archived) rather than permanently removed. Archived cases can be viewed in a dedicated archive view and restored at any time.

---

## Manual Test Execution (Test Collections)

Test collections are groups of test cases assembled for manual execution. They enable structured manual testing workflows with user assignment, step-level result tracking, and progress reporting.

### Creating a Collection

1. Create a test collection with a name, description, and optionally assign it to a user
2. Add test cases to the collection (individually or in bulk)
3. Optionally link the collection to a release for progress tracking
4. Organize collections in folders for better structure

### Execution Flow

1. Assigned user opens the collection and begins executing cases
2. Each case starts as **not_executed**
3. User marks cases as **in_testing** when starting
4. For each case, the user can record step-level results (pass/fail per step)
5. User sets final status: **passed**, **failed**, or **blocked**
6. For failures, a **caused_by** field can record a short issue note
7. Duration and notes can be recorded per execution

### Execution Statuses

| Status | Description |
|--------|-------------|
| not_executed | Case has not been started |
| in_testing | Currently being executed |
| passed | All steps passed |
| failed | One or more steps failed |
| blocked | Cannot be executed (dependency or environment issue) |

### Step-Level Results

Each step within a case can be independently marked with a status and an issue description. This enables detailed failure tracking- you can see exactly which step failed and why, including steps inherited from parent/dependent cases.

### Collection Properties

| Field | Description |
|-------|-------------|
| Name | Collection name |
| Description | Purpose of this test run |
| Assigned To | User responsible for execution |
| Status | Overall collection status |
| Priority | Execution priority |
| Environment | Target environment (e.g., staging, production) |
| Start/End Date | Planned execution window |
| Release | Optional link to a release for progress tracking |

---

## Agentic Test Execution (an AI agent runs the manual run)

Most AI in test management stops at drafting: it generates test cases, and a human runs them. M00N Report also exposes the manual run itself as something an AI agent can operate over MCP, so the agent can do the testing and record the result, step by step, on the same execution a human would have used.

The run is data, not a UI. Every case-run row is addressable, its steps arrive as structured fields, and every result the agent writes reads back the same way - which is what makes an execution drivable by a program at all.

### Why this is different from AI test case generation

| | Case generation | Agentic execution |
|---|---|---|
| What the AI produces | Draft cases for a human to run | Actual run results |
| Who executes the steps | A human tester | The agent, with its own tooling |
| What lands in the TMS | Test case definitions | Per-step pass/fail, notes, failure reasons, statuses |
| Evidence trail | The human's | Identical shape, attributed to the agent's account and the MCP audit log |

### The loop

1. **Start:** the `run_manual_execution` prompt lists the project's runnable executions and hands the agent the whole procedure.
2. **Read the vocabulary:** `get_project` returns the project's configurable status values (`execution_status`, `test_case_execution_status`, `test_run_status`). Statuses are per-project, so an agent never hardcodes them. Defaults: not_executed / in_testing / passed / failed / blocked.
3. **Mark it started:** `update_execution` with status `in_testing`.
4. **List the work:** `get_execution_cases` returns each case-run row with its `executionCaseId`, its `caseId`, its group, its current status and its `steps` - 0-based index, action, expected result, current step status, existing issue description. External ticket links come back too, when the key also holds `integrations:read`.
5. **Do the testing:** the agent exercises the described steps against the system under test with tooling it already has - a browser MCP such as Playwright, an HTTP client, a terminal, a device farm. M00N Report supplies the run, the vocabulary and the write-back; the customer's own agent supplies the doing. There is no built-in AI in the product and nothing is sent to a separate AI service.
6. **Report:** `report_case_result`, atomically and idempotently, per case.
7. **Close:** `update_execution` with the terminal status - passed, failed or blocked.

### What report_case_result writes

| Field | Meaning |
|-------|---------|
| executionCaseId | The case-run row (from `get_execution_cases`). Alternative: executionId + numeric caseId, plus groupId when the case sits in several groups |
| steps[].index | 0-based position in the case steps |
| steps[].status | not_executed, passed, failed, blocked or skipped |
| steps[].issueDescription | Note, defect summary or ticket link for a failed step |
| status | An explicit case status, or `"auto"` to derive it from the steps, or omitted to leave it unchanged while writing a single step |
| causedBy | Which step or condition caused the failure |
| notes | Free-text notes for the case run |
| durationMinutes | Time spent on the case |

It returns the resulting case status, the group status and fresh execution progress, so an agent tracks completion without extra calls.

### Statuses derive themselves

- **Case:** `status: "auto"` computes the case result from its steps, in precedence order - failed, then blocked, then in_testing, then passed.
- **Source group** (a collection, suite or named case group inside the execution): rolls up automatically as results arrive. All cases not executed is not_started; some executed is in_progress; all executed is completed, or blocked when a finished group still holds a blocked case. `set_execution_source_status` exists for a manual override and is recomputed by the next report on that group.
- **Execution:** set explicitly by the agent at the end with `update_execution`.

### Properties that make it safe to hand to an agent

- **Governed, not a side channel.** The agent acts as the account its MCP key belongs to, under that account's role and project permissions, and every call lands in the MCP audit log with a UI in the app. An agent can never exceed the permissions of the user whose key it holds.
- **Idempotent.** Re-reporting the same case or step converges instead of duplicating, so a retried or resumed run is safe.
- **Case snapshotting still applies.** A case edited during the run does not rewrite an execution already in flight.
- **Readable back by humans.** Everything the agent writes appears in the UI as ordinary execution evidence: the same per-step pass/fail, notes and failure reasons a human tester would have left.
- **Mixed runs are normal.** An agent can execute part of an execution and a human the rest, or a human can take over mid-run - both write to the same case-run rows.
- **Failures link out.** `add_external_link` attaches a Jira, Linear, GitHub or GitLab ticket to the case or the execution in the same pass.

### Where it fits: regression and exploratory testing

Two workloads carry most of the value, and they feed each other.

**Regression testing** is the strongest fit. The same scripted pass runs every release - often cases too UI-heavy or too short-lived to be worth automating properly. An agent follows the written steps rather than habit, so the run is executed the same way every time, does not lose attention on case 60, and produces evidence of the same shape release after release, which is what makes two runs comparable at all. Typical use: hand the agent the regression execution overnight and spend the morning on the failures.

**Exploratory testing** is the other. Give the agent a charter instead of a checklist - "spend 30 minutes on the new checkout flow, try what a confused user would try" - and it probes the app the way a tester would. The difference from an exploratory session in a chat window is what survives it: the agent writes findings up as real test cases with `create_test_case`, files defects with `add_external_link`, and adds the worthwhile ones to an execution. Exploration turns into regression coverage instead of evaporating.

The loop closes: an exploratory session produces cases, those cases become the next regression execution, the agent runs that execution, and the tester goes exploring somewhere new.

Adjacent fits: a smoke run against every environment (same execution, different target, comparable results); proving a freshly written case is followable before automating it; and splitting one execution so the mechanical cases go to the agent and the judgement calls to a human.

### Example instruction to an agent

> Run the "Sprint 48 Smoke" execution against staging: follow each case's steps in the browser, record pass/fail per step, mark blocked cases with the reason, link failures to Jira, and set the final execution status.

Docs: https://m00nreport.com/documentation/mcp/agentic-testing

---

## Release Management

Releases aggregate test results from both manual test collections and automated test launches, providing a unified view of release readiness.

### Release Structure

| Field | Description |
|-------|-------------|
| Name | Release name (e.g., "v2.4.0") |
| Version | Version identifier |
| Description | Release notes or scope |
| Status | planned, in_progress, or completed (customizable) |
| Start/End Date | Release timeline |

### Linking Test Results

Releases combine two sources of test data:

**Manual Executions:** Link test collections to a release. The release dashboard shows aggregated progress across all linked collections- how many cases passed, failed, are blocked, or still need execution.

**Automated Launches:** Link CI/CD test runs (launches) to a release. The release shows automated test results including pass rates and specific failures.

### Release Progress Dashboard

The release details page provides:

- **Manual test stats:** Total cases, passed, failed, blocked, in testing, not executed- aggregated across all linked collections
- **Automated test stats:** Total tests, passed, failed, skipped- aggregated across all linked launches
- **Progress view:** Lists all failed, in-testing, and blocked test cases with details (case ID, title, assigned user, collection name, step-level issues)
- **Failed automated tests:** Lists specific test failures from linked launches

### Release Workflow

1. Create a release with name, version, and dates
2. Optionally link to Jira sprints for project tracking
3. Link manual test collections for manual QA progress
4. Link automated test launches for CI/CD results
5. Monitor the progress dashboard as testing proceeds
6. View failed and blocked tests to identify remaining work
7. Mark release as completed when all criteria are met

### Jira Integration

Releases can be linked to Jira sprints (scrum or kanban boards) for cross-tool project tracking. Sprint data includes board type, project key, and direct links back to Jira.

### Vocabulary mapping

For readers coming from another test management tool:

| M00N Report | Elsewhere |
|---|---|
| Test Execution | Test Run, Test Plan, Test Cycle |
| Test Collection | Suite, reusable Cycle |
| Release | Milestone, Version |
| Launch | Automated test run / CI run |
| Folder + Suite | Test repository tree |

---

## MCP Server (AI Integration)

**Package:** `@m00nsolutions/mcp-server`
**Install:**

```bash
npm install @m00nsolutions/mcp-server
# Or use npx for one-time execution
npx -y @m00nsolutions/mcp-server
```

**Cursor IDE Configuration (.cursor/mcp.json):**

```json
{
  "mcpServers": {
    "m00n": {
      "command": "npx",
      "args": ["-y", "@m00nsolutions/mcp-server"],
      "env": {
        "M00N_API_URL": "https://m00nreport.com",
        "M00N_API_KEY": "m00n_mcp_your_key_here"
      }
    }
  }
}
```

**Claude Desktop Configuration:**

```json
{
  "mcpServers": {
    "m00n": {
      "command": "npx",
      "args": ["-y", "@m00nsolutions/mcp-server"],
      "env": {
        "M00N_API_URL": "https://m00nreport.com",
        "M00N_API_KEY": "m00n_mcp_your_key_here"
      }
    }
  }
}
```

**Environment Variables:**

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| M00N_API_URL | Yes | - | M00N Report API URL |
| M00N_API_KEY | Yes | - | MCP API key (starts with `m00n_mcp_`) |
| M00N_INSECURE_SSL | No | false | Skip SSL verification (self-hosted) |
| M00N_DEBUG | No | false | Enable verbose logging |
| M00N_TIMEOUT_MS | No | 30000 | Request timeout in milliseconds |

**Capabilities:**

- 60 tools - 34 state-changing, 26 read-only, 10 flagged destructive: test case authoring, folder and suite management, test collections, manual executions, releases, external links, coverage gaps, project health and run analytics
- 8 resources: Projects, project health, flaky tests, failing tests, folder tree, launch reports, test cases, release summaries
- 7 prompts: analyze_flaky_tests, debug_test_failure, release_readiness, generate_test_cases, weekly_health_report, investigate_regression, run_manual_execution

**Transports and auth:** the `@m00nsolutions/mcp-server` npm package runs locally over stdio and authenticates with an MCP key (`m00n_mcp_...`), and works with any MCP client. A hosted remote connector is also available at `POST /mcp` (Streamable HTTP JSON-RPC) using OAuth 2.1 with PKCE and dynamic client registration; its redirect allowlist currently covers claude.ai and claude.com only, so every other client uses the stdio package with a key.

**Scope notes (so these are not overstated):** `suggest_test_cases` does not call an LLM - it returns project-aware scaffolding from templates, the customer's own model does the generating, and `create_test_case` persists the result. `get_feature_scan` requires a connected Jira integration. `delete_release`, `delete_folder` and `delete_test_case` archive (soft-delete) rather than permanently remove. `get_coverage_gaps` is Jira-link plus bidirectional autotest gap analysis, not a requirements traceability matrix.

---

## Playwright Reporter

**Package:** `@m00nsolutions/playwright-reporter`
**Install:**

```bash
npm install @m00nsolutions/playwright-reporter --save-dev
```

**Configuration (playwright.config.ts):**

```typescript
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['list'],
    ['@m00nsolutions/playwright-reporter', {
      serverUrl: 'https://m00nreport.com',
      apiKey: process.env.M00N_API_KEY,
      launch: 'Regression Suite',
      tags: ['smoke', 'regression'],
    }],
  ],
});
```

**Options:**

| Option | Type | Required | Description |
|--------|------|----------|-------------|
| serverUrl | string | Yes | M00N Report server URL |
| apiKey | string | Yes | Project API key |
| enabled | boolean | No | Enable/disable reporting (default: true) |
| launch | string | No | Launch name for grouping runs |
| tags | string[] | No | Tags for filtering |
| attributes | object | No | Custom key-value metadata |

**Environment Variables:** `M00N_SERVER_URL`, `M00N_API_KEY`, `M00N_LAUNCH`, `M00N_TAGS`, `M00N_ENABLED`

**Features:** Real-time step streaming, automatic screenshot/video/trace capture, retry tracking, CI/CD auto-detection (GitHub Actions, GitLab CI, Jenkins, Bitbucket, Azure DevOps, CircleCI, Travis CI).

---

## Jest Reporter

**Package:** `@m00nsolutions/jest-reporter`
**Install:**

```bash
npm install @m00nsolutions/jest-reporter --save-dev
```

**Configuration (jest.config.js):**

```javascript
module.exports = {
  reporters: [
    'default',
    ['@m00nsolutions/jest-reporter', {
      serverUrl: 'https://m00nreport.com',
      apiKey: process.env.M00N_API_KEY,
      launch: 'API Tests',
      tags: ['api', 'unit'],
    }],
  ],
};
```

**Options:**

| Option | Type | Required | Description |
|--------|------|----------|-------------|
| serverUrl | string | Yes | M00N Report server URL |
| apiKey | string | Yes | Project API key |
| launch | string | No | Launch name for grouping runs |
| tags | string[] | No | Tags for filtering |
| attributes | object | No | Custom key-value metadata |
| debug | boolean | No | Enable debug logging (default: false) |
| realtime | boolean | No | Enable real-time streaming (default: true) |
| includeConsole | boolean | No | Capture console output (default: false) |
| includeCoverage | boolean | No | Include coverage data (default: false) |
| includeSnapshots | boolean | No | Track snapshot changes (default: true) |
| slowTestThreshold | number | No | Slow test threshold in ms (default: 5000) |

**Environment Variables:** `M00N_SERVER_URL`, `M00N_API_KEY`, `M00N_LAUNCH`, `M00N_TAGS`, `M00N_DEBUG`

**Features:** Real-time streaming, retry tracking, jest-circus support, snapshot tracking, watch mode, parallel execution, console output capture.

---

## Pytest Reporter

**Package:** `m00nreport-pytest` (PyPI, MIT-licensed)
**Install:**

```bash
pip install m00nreport-pytest
```

The package registers itself as a pytest plugin via the standard `pytest11` entry point, so pytest discovers and loads it automatically once installed - no `conftest.py` wiring, `-p` flag, or config block needed.

**Configuration (fastest path - environment variables):**

```bash
export M00N_SERVER_URL=https://m00nreport.com   # or your self-hosted URL
export M00N_API_KEY=m00n_xxxxxxxxxxxxx           # your project API key

pytest
```

**Configuration (pytest.ini, checked into the repo):**

```ini
[pytest]
m00n_server_url = https://m00nreport.com
m00n_api_key = m00n_xxxxxxxxxxxxx
m00n_launch = Nightly Regression
m00n_tags = smoke, api
```

Never commit a real API key in `pytest.ini` - keep the key in `M00N_API_KEY`. Precedence, highest wins: CLI flag > `pytest.ini` key > environment variable > default.

**Key APIs:**

| API | Purpose |
|-----|---------|
| `from m00n_reporter import step, attach` | `with step("name"): ...` records a named, timed step (supports nesting); `attach("file.png")` attaches a file. |
| `@pytest.mark.m00n(case_id=42, tags=[...])` | Links a test to an existing M00N Report test case. The `m00n` marker is auto-registered by the plugin - do not add it to `pytest.ini`'s `markers` list. |
| Playwright `page` fixture | Calls made through pytest-playwright's `page` fixture are auto-captured as `pw:api` steps, with a screenshot on failure. Requires both `pytest-playwright` AND `playwright` installed - `pip install m00nreport-pytest[playwright]` pulls in `pytest-playwright` for you; browsers still need a separate `python -m playwright install chromium`. Otherwise this layer is fully inert. |
| `pip install pytest-rerunfailures` + `pytest --reruns 3` | Each retry attempt reports as a distinct entry with its own `testId` and an incrementing zero-based `retry` index. |
| `--m00n-debug` / `M00N_DEBUG=true` | Verbose `[m00nreport]` logging of every HTTP retry attempt and any final exhaustion. |

CI metadata (branch, commit, pipeline, build number/URL) is auto-detected with zero configuration on GitHub Actions, GitLab CI, Jenkins, Bitbucket Pipelines, Azure DevOps, CircleCI, and Travis CI. `pytest-xdist` parallel execution (`-n auto`) is supported and reports as one launch.

---

## NUnit Reporter

**Package:** `M00nReport.NUnit` (NuGet, MIT-licensed) + optional `M00nReport.NUnit.Playwright`
**Install:**

```bash
dotnet add package M00nReport.NUnit
# Only if the project already uses Microsoft.Playwright.NUnit:
dotnet add package M00nReport.NUnit.Playwright
```

**Activate with one attribute:**

```csharp
// AssemblyInfo.cs (or any file in the test project)
[assembly: M00nReport.NUnit.M00nReport]
```

No `[SetUpFixture]`, no `ITestEventListener`, no runsettings entry - the attribute is an NUnit `ITestAction` that wires itself into the run automatically.

**Configuration (fastest path - environment variables):**

```bash
export M00N_SERVER_URL=https://m00nreport.com   # or your self-hosted URL
export M00N_API_KEY=m00n_xxxxxxxxxxxxx           # your project API key

dotnet test
```

**Configuration (m00nreport.json, next to the test assembly's build output):**

```json
{
  "serverUrl": "https://m00nreport.com",
  "apiKey": "m00n_xxxxxxxxxxxxx",
  "launch": "Nightly Regression",
  "tags": ["smoke", "api"]
}
```

Never commit a real API key - keep it in `M00N_API_KEY`. Precedence, highest wins: environment variable > `m00nreport.json` > built-in default.

**Key APIs:**

| API | Purpose |
|-----|---------|
| `M00nStep.Run("name", () => { ... })` | Records a named, timed step; a thrown exception marks it `failed`, a normal return marks it `passed`. |
| `M00nAttach.Bytes(bytes, "name.png", "image/png")` / `M00nAttach.File(path)` | Attach a file to the current test. |
| `[M00nCaseId(42, "regression", "checkout")]` | Links a test to an existing M00N Report test case. |
| `[Retry(3)]` (NUnit built-in) | Each attempt reports as a distinct entry with its own `testId` and a zero-based `retry` index (`TestContext.CurrentContext.CurrentRepeatCount`). |
| `M00nPageTest` base class (from `M00nReport.NUnit.Playwright`) | Change ONLY the base class from `PageTest` to `M00nPageTest` - existing `Page.<Method>(...)` calls keep working and a curated set of navigation/interaction methods is recorded as `pw:api` steps, with an automatic failure screenshot. Set `M00N_TRACING=retain-on-failure` and/or `M00N_VIDEO=retain-on-failure` for trace/video too. |
| `M00N_DEBUG=true` / `"debug": true` | Verbose `[m00nreport]` logging. |

Note: NUnit exposes no source file path at runtime, so `filePath` for this reporter is the test's namespace-qualified class name, not a real file path - this is expected, not a bug. CI metadata is auto-detected with zero configuration on the same 7 major providers as the pytest reporter.

---

## Build Your Own Integration (Ingest API v2)

Every M00N Report reporter, including the official Playwright, Jest, Pytest, JUnit 5, and NUnit ones, speaks the
same small HTTP API: the ingest v2 protocol. If there is no official reporter for a given
language or test runner, a custom integration can be built against this API directly, in any
language that can send HTTP requests.

**The contract:** the full OpenAPI 3.0 specification is public and machine-readable at
`YOUR_SERVER_URL/api/ingest/v2/openapi.json` (e.g. `https://m00nreport.com/api/ingest/v2/openapi.json`)
- it is the single source of truth for every field, type, limit, and enum. A human-browsable
version is at `https://m00nreport.com/documentation/integration-api/reference`, and the full guide
is at `https://m00nreport.com/documentation/integration-api/guide`.

**Authentication:** every request except the two health checks carries an `X-API-Key` header
(format `m00n_...`). The key alone identifies both the organization and the project - never send
a separate project id.

**Protocol, in order:**

1. `POST run/start` once per run - the server creates the run and returns `runId` (UUID). Every
   later call echoes this `runId`.
2. `POST test/start` per test attempt, fire-and-forget - the CLIENT generates `testId` (UUID v4)
   so steps can stream immediately without waiting for a response.
3. `POST steps/stream` batches of live step events (optional but recommended for real-time UX).
   Batches may mix tests and runs; max 1000 items per call.
4. `POST attachment/upload` (small files, buffered multipart, <=10MB), or the
   `attachment/presign` + PUT + `attachment/confirm` trio for large ones, which sends the bytes
   straight to storage instead of through the service. `attachment/stream` (same multipart shape
   as upload) is the fallback when direct upload is unavailable or the PUT fails.
5. `POST test/complete` per test attempt with final status, error, timings, any remaining steps,
   and attachment descriptors.
6. `POST run/end` once, last.

**Retries:** on a retry, send `test/start` and `test/complete` again with the SAME `titlePath`, a
NEW `testId`, and `retry` incremented (0-based: first attempt is `retry: 0`, first retry is
`retry: 1`, and so on). The dashboard groups every attempt sharing a `titlePath` into one
flaky-test history.

**Linking a result to a manual test case:** three channels, merged, highest precedence first.
`caseId` / `caseIds` in the body are internal case ids and are used as sent. `annotations.caseId`
/ `annotations.caseIds` carry the same internal ids, for runners whose annotation API is the only
place a test can hold metadata. A `[TC-42]` marker anywhere in `titlePath` uses the per-project
case NUMBER the UI shows, not the internal id, and is resolved against the run's project
server-side; a number matching nothing links nothing and is not an error. A `[tag: NAME]` marker
in `titlePath` adds a tag the same way.

**titlePath semantics:** `titlePath` is the single field the dashboard uses to derive a test's
title, its suite grouping, its FILE badge, and its browser badge. Shape:
`[file-or-module, ...suite/describe/class names, test title]`.

- The LAST element is always the display title.
- Elements between the first and last group the test into suites; the element immediately before
  the title is shown as the closest enclosing suite.
- The FIRST element becomes the FILE badge only when it looks like a file (a path separator, or a
  trailing extension like `.spec.ts`) - a bare namespace/module segment (e.g. NUnit's root
  namespace) does not trigger it.
- A trailing `[chromium]` / `[firefox]` / `[webkit]` suffix ON THE TITLE STRING (not a separate
  array element) renders a browser badge.
- Never put the run/launch name inside `titlePath` - that belongs in `run/start`'s `launch` field.

Examples: Playwright JS raw `test.titlePath()`: `['', 'chromium', 'auth.spec.ts', 'Login', 'valid credentials']`.
pytest: `['tests/test_login.py', 'test_valid_credentials[chromium]']`.
NUnit: `['MyApp', 'Tests', 'LoginTests', 'ValidCredentials']`.

**Limits:** 5000 steps and 100 attachments per `test/complete`; 1000 items per `steps/stream`
batch; 200MB per file; 10 files per multipart request; use `attachment/upload` (buffered) for
files <=10MB and `attachment/stream` for files >10MB to avoid buffering large files in memory;
per-run attachment byte quota by pricing tier (413 `RUN_ATTACHMENT_LIMIT_EXCEEDED` when exceeded).

**Health:** two unauthenticated liveness probes are part of the contract: `GET /healthz` and
`GET /api/ingest/health` (identical response shape, `{ ok: true, service: "ingest" }`). Probe
`/api/ingest/health` first, then `/healthz`, and treat the result as ADVISORY: it words the
failure message, it does not stop reporting. Assert `ok === true` in the response BODY, never
just the status code - a CDN that rewrites an unrouted path to an SPA shell answers 200 with
HTML, so a status-only probe passes against a service it never reached. On the m00nreport.com
cloud apex only `/api/ingest/health` is reachable (only path prefixes such as `/api/*` reach the
backend there); self-hosted deployments, local development and direct ingest URLs serve
`/healthz`. Any server implementing this contract should serve both paths.

**Resilience (the iron rule):** a reporter must NEVER fail the test run and must never change its
exit code. Wrap all network calls in try/catch, retry 5xx responses with exponential backoff,
stop attempting further calls after repeated failures, and degrade silently (log a warning) if
the server stays unreachable. Three details that are easy to get wrong:

- **Do not retry a permanent error.** Responses carry a `code`; the permanent ones (bad key,
  inactive subscription, unknown run, file too large, quota exhausted) will never succeed, and
  retrying each of them costs three attempts and two backoff sleeps on every call for the whole
  suite. The full enum is in the OpenAPI document.
- **`test/start` and `steps/stream` are fire-and-forget.** Retry them only if your runner makes
  that free: the service upserts, so losing one costs the live row between start and completion,
  not the result. A retried background POST can also overtake `run/end`.
- **Refuse redirects.** HTTP clients strip `Authorization` across an origin but do not strip
  custom headers, and this API authenticates with `X-API-Key`. Following a redirect hands the
  project key to whoever answers for the configured host. Likewise, escape CR, LF and quotes out
  of any filename you place in a multipart `Content-Disposition` header.

**Conformance:** a reporter's traffic is expected to satisfy 15 named invariants -
`AUTH_HEADER_PRESENT`, `RUN_START_FIRST`, `RUN_ID_ECHO`, `TESTID_UUID`, `START_BEFORE_COMPLETE`,
`COMPLETE_ALL_STARTED`, `RETRY_MONOTONIC`, `RUN_END_ONCE`, `RUN_END_LAST`, `PRESIGN_BEFORE_CONFIRM`,
`PUT_BEFORE_CONFIRM`, `CONFIRM_ID_FROM_PRESIGN`, `SCHEMA_VALID`, `MULTIPART_FIELDS`, and (under
fault injection) `RETRIES_ON_FAILURE`. Fourteen apply to every run; the fifteenth needs injected
faults. The conformance suite that checks them lives in the M00N Report repository
(`reporter-conformance/`).

### Compact endpoint reference

All paths below are relative to the ingest server root (e.g. `https://m00nreport.com` or a
self-hosted URL). Full field-level types, enums, and descriptions are in the OpenAPI document.

| Method + path | Required fields | Response fields |
|---|---|---|
| `GET /healthz` | (none, no auth) | `ok`, `service` |
| `GET /api/ingest/health` | (none, no auth) | `ok`, `service` |
| `POST /api/ingest/v2/run/start` | `launch` | `ok`, `runId` |
| `POST /api/ingest/v2/test/start` | `runId`, `testId`, `titlePath` | `ok`, `testId`, `displayId` |
| `POST /api/ingest/v2/steps/stream` | `items` (array of StepItem, max 1000) | `ok`, `count` |
| `POST /api/ingest/v2/test/complete` | `testId`, `runId`, `status` | `ok` |
| `POST /api/ingest/v2/run/end` | `runId` | `ok` |
| `POST /api/ingest/v2/attachment/upload` (multipart) | `runId`, `testId`, `file` | `ok`, `attachments[]` (AttachmentResult) |
| `POST /api/ingest/v2/attachment/stream` (multipart) | `runId`, `testId`, `file` | `ok`, `attachments[]` (AttachmentResult) |
| `POST /api/ingest/v2/attachment/presign` | `runId`, `testId`, `filename`, `contentType`, `size` (all required; `size` exact, 1..209715200) | `url`, `attachmentId`, `objectName`, `bucket`, `expiresIn` |
| `POST /api/ingest/v2/attachment/confirm` | `runId`, `attachmentId`, `filename`, `contentType` | `ok`, `attachmentId`, `objectName`, `size` |

`test/start` and `test/complete` both also accept the optional fields `filePath`, `retry`,
`startedAt` (and `test/complete` additionally: `titlePath`, `annotations`, `endedAt`, `duration`,
`error`, `steps[]`, `attachments[]`). `run/start` also accepts optional `tags[]`, `total`,
`startedAt`, `attributes`, `metadata`. `run/end` also accepts optional `status`, `endedAt`.

StepItem (used in `steps/stream`'s `items[]`): required `runId`, `testId`, `title`; optional
`action` (`begin`|`end`), `status`, `category`, `duration`, `error`, `nestingLevel`, `stepIndex`.
EmbeddedStep (used in `test/complete`'s `steps[]`, a DIFFERENT shape - no `runId`/`testId` per
item): all fields optional, `title`, `category`, `status`, `duration`, `error`, `nestingLevel`,
`index` (falls back to array position if omitted), `stepIndex` (accepted as an alias for `index`).

### AI prompt: build a reporter for any language or test runner

The following is a complete, self-contained prompt for an AI coding assistant to build a custom
M00N Report integration in any language or test runner. It is also available as a copy-paste
Collapse block on the "Build your own integration" guide page in the app
(`https://m00nreport.com/documentation/integration-api/guide`).

NOTE: keep this prompt in sync with the `AI_REPORTER_BUILDER_PROMPT` constant in
`src/components/docs/sections/integrations/BuildYourOwnIntegration.jsx`.

# Build a M00N Report reporter for MY_TEST_RUNNER

You are an AI assistant helping me build a custom M00N Report integration for a test runner or
language that does not have an official M00N Report reporter yet. Follow this guide end to end.
Do not skip the resilience requirements - they are acceptance criteria, not suggestions.

## Before you write any code, ask me for

1. `YOUR_SERVER_URL` - the M00N Report server URL (e.g. `https://m00nreport.com`, or a
   self-hosted URL).
2. `YOUR_API_KEY` - a project API key, format `m00n_...`.
3. `YOUR_TEST_RUNNER_NAME_AND_LANGUAGE` - the exact test framework, language, and version
   (e.g. "Go testing package", "RSpec 3.12", "Robot Framework 7").
4. `YOUR_HOOK_MECHANISM` - how that runner exposes lifecycle hooks (listener interface, plugin
   API, reporter interface) so results can be reported without editing every individual test file.

Wait for my answers before generating code.

## Step 0 - Read the contract

Fetch `YOUR_SERVER_URL/api/ingest/v2/openapi.json` (a public, unauthenticated GET). It is the
single source of truth for every field name, type, limit, and enum below. If anything below
conflicts with the fetched document, the document wins.

## Authentication

Every request below except the two health checks carries:

```
X-API-Key: YOUR_API_KEY
```

The key alone identifies both the organization and the project - never send a separate project id.

## Lifecycle: 6 core endpoints, in this order

1. `POST YOUR_SERVER_URL/api/ingest/v2/run/start` - ONCE, first.
   Body: `{ launch (string, required), tags (string[]), total (number), startedAt (ISO 8601), attributes (object), metadata (object) }`.
   Response: `{ ok, runId }` - store `runId`, echo it in every later call.

2. `POST YOUR_SERVER_URL/api/ingest/v2/test/start` - once per test attempt, fire-and-forget (do
   not block the test on the response).
   Body: `{ runId, testId, titlePath (array, required - see titlePath rules below), filePath, retry, startedAt }`.
   YOU (the client) generate `testId` as a fresh UUID v4 - never reuse one, even across a retry.

3. `POST YOUR_SERVER_URL/api/ingest/v2/steps/stream` - optional but recommended, batched.
   Body: `{ items: StepItem[] }`, max 1000 items per call.
   StepItem: `{ runId, testId, title (required), action ('begin'|'end'), status, category, duration (ms), error, nestingLevel (0-based depth), stepIndex (stable per-test index) }`.
   Recommended cadence: flush every ~200ms or every ~100 buffered items, whichever comes first -
   keeps the dashboard live without hammering the server.
   GOTCHA: this endpoint's field is `stepIndex`. The separate steps array embedded inside
   `test/complete` (below) uses `index` instead, falling back to array position if omitted, with
   `stepIndex` accepted only as an alias there. Do not mix the two field names up between the two
   call sites.

4. Attachments - upload each file with ONE of these three paths:
   - `POST .../attachment/upload` (multipart form: `runId`, `testId`, one or more `file` parts) -
     buffered in memory, use for files <=10MB.
   - `POST .../attachment/stream` (same multipart shape) - streamed to storage without buffering,
     use for files >10MB.
   - OR `POST .../attachment/presign` `{ runId, testId, filename, contentType, size }` ->
     `{ url, attachmentId, expiresIn }`, PUT the bytes to `url`, then `POST .../attachment/confirm`
     `{ runId, attachmentId, filename, contentType }`. Preferred for large files: the bytes never
     pass through the service. `run/start` reports `directUpload` and `directUploadMinBytes`.
     - `size` is required and exact. It is signed into the URL as the Content-Length your PUT must
       send, along with the content type. Any other length, or a chunked request with no
       Content-Length, is refused by storage and nothing is stored.
     - Send NO API key on the PUT: the signature is the authorization.
     - Nothing is recorded until confirm. Confirm is safe to retry, and if its response is lost you
       must retry it rather than re-uploading. Do NOT fall back to `attachment/stream` once the PUT
       succeeded, or the file is stored and charged twice.
     - Fall back to `attachment/stream` when presign or the PUT fails, choosing on the outcome and
       never on a storage error code: S3 answers `SignatureDoesNotMatch` where MinIO answers
       `MissingContentLength` for the same mistake.
     - A file above the per-file cap answers 400 `ATTACHMENT_TOO_LARGE`. Skip that one file and
       carry on; it is not the run-level quota and must not stop later attachments.
   Hard caps: 200MB per file, 10 files per multipart request, plus a per-run attachment byte quota
   (413 `RUN_ATTACHMENT_LIMIT_EXCEEDED` when exceeded - treat this as non-fatal: log a warning and
   continue the run).

5. `POST YOUR_SERVER_URL/api/ingest/v2/test/complete` - once per test attempt, after its `test/start`.
   Body: `{ testId, runId, status (required, enum: passed|failed|skipped|timedOut|timedout|interrupted), titlePath, filePath, annotations, retry, startedAt, endedAt, duration (ms), error, steps[] (EmbeddedStep - any steps not already sent via steps/stream), attachments[] (EmbeddedAttachment, base64 inline - leave empty if files were uploaded separately in step 4, which is the default and preferred path) }`.

6. `POST YOUR_SERVER_URL/api/ingest/v2/run/end` - EXACTLY ONCE, last, after every `test/complete`.
   Body: `{ runId (required), status (enum: passed|failed|timedOut|timedout|interrupted|finished|completed), endedAt }`.

## testId / runId rules

- `runId` is server-generated, returned once by `run/start`. Every later call in the run echoes
  that SAME `runId`.
- `testId` is a client-generated UUID v4, one per TEST ATTEMPT (a retried test gets a NEW
  `testId` per attempt, not one shared `testId`). It is generated client-side specifically so
  `steps/stream` can start before `test/start`'s response comes back.

## Retry semantics

On a retry: keep the SAME `titlePath`, generate a NEW `testId`, and set `retry` to the previous
attempt's `retry` plus 1 (0-based, so a test's first attempt is `retry: 0`, its first retry is
`retry: 1`, and so on). The dashboard groups every attempt sharing a `titlePath` into one
flaky-test history.

## titlePath rules

`titlePath` is an array: `[file-or-module, ...suite/describe/class names, test title]`.

- The LAST element is always the test's display title.
- Elements between the first and the last group the test into suites in the dashboard; the
  element immediately before the title is shown as its closest enclosing suite.
- The FIRST element becomes the dashboard's FILE badge only when it looks like a file (contains a
  path separator, or ends in a short extension such as `.spec.ts`) - a bare namespace or module
  segment does not trigger it.
- A trailing `[chromium]` / `[firefox]` / `[webkit]` suffix ON THE TITLE STRING ITSELF (not a
  separate array element) renders a browser badge.
- Never put the run/launch name inside `titlePath` - that belongs in `run/start`'s `launch` field.

Examples from existing ecosystems (adapt the shape to your runner, do not copy verbatim):

- Playwright JS raw `test.titlePath()`: `['', 'chromium', 'auth.spec.ts', 'Login', 'valid credentials']`
- pytest: `['tests/test_login.py', 'test_valid_credentials[chromium]']`
- NUnit: `['MyApp', 'Tests', 'LoginTests', 'ValidCredentials']`

## Resilience - ACCEPTANCE CRITERIA, not suggestions

1. The reporter must NEVER fail or crash the test run. Wrap every network call in try/catch (or
   your language's equivalent) - a reporting failure is a logged warning, never an exception that
   reaches the test framework.
2. Retry every request that receives a 5xx response, INCLUDING `test/start` (its fire-and-forget
   nature does not exempt it from retry-with-backoff), using exponential backoff.
3. Implement a circuit breaker: after N consecutive failures (e.g. 3-5), stop attempting further
   ingest calls for the rest of the run and degrade silently, rather than retrying forever and
   delaying every test.
4. Treat `GET YOUR_SERVER_URL/api/ingest/health` (no auth required) as an ADVISORY pre-flight
   probe only. An unreachable or non-200 response is informational, not a hard blocker - it is
   reasonable (and is what the official Playwright JS reporter does) to disable the reporter for
   the run when the probe fails, but never let the probe itself throw or delay tests.

## Definition of done - 15 conformance invariants

A conformant reporter's traffic passes all 15 named checks the M00N Report conformance suite runs
against a recorded session. Use them as your test plan even if the conformance package is not
attached to this project directly:

1. `AUTH_HEADER_PRESENT` - every request carries `X-API-Key` starting with `m00n_`
2. `RUN_START_FIRST` - `run/start` is the first ingest call
3. `RUN_ID_ECHO` - every `runId` used was previously issued by `run/start`
4. `TESTID_UUID` - every `testId` is a valid UUID
5. `START_BEFORE_COMPLETE` - `test/start` precedes `test/complete` for the same `testId`
6. `COMPLETE_ALL_STARTED` - every started test is eventually completed
7. `RETRY_MONOTONIC` - `retry` starts at 0 and increments by 1 per attempt of the same `titlePath`
8. `RUN_END_ONCE` - `run/end` is called exactly once per run
9. `RUN_END_LAST` - no ingest calls happen for a run after its `run/end`, except `attachment/confirm`, which is allowed to arrive late because the object is already stored
10. `PRESIGN_BEFORE_CONFIRM` - every `attachment/confirm` follows an `attachment/presign`
11. `PUT_BEFORE_CONFIRM` - every `attachment/confirm` follows an upload to the presigned URL
12. `CONFIRM_ID_FROM_PRESIGN` - confirm only uses `attachmentId` values the server issued
13. `SCHEMA_VALID` - every JSON body validates against the OpenAPI request schema for its path
14. `MULTIPART_FIELDS` - multipart uploads carry `runId` and `testId` fields
15. `RETRIES_ON_FAILURE` - any request that received an injected or transient failure was retried
    and eventually succeeded

## Step-by-step

1. Ask me for `YOUR_SERVER_URL`, `YOUR_API_KEY`, and the runner specifics above; wait for my answer.
2. Fetch and read the OpenAPI document.
3. Identify the runner's hook/listener/plugin mechanism for: run start, per-test start, per-step
   events (if the runner exposes them), per-test finish, and run end.
4. Implement the 6 calls above behind that mechanism, generating `testId`/`runId` per the rules
   above.
5. Implement the resilience acceptance criteria: retry with backoff, circuit breaker, never-fail.
6. Wire up attachment upload for screenshots, logs, or traces the runner already captures, if any.
7. Run a small suite - including at least one intentionally failing test and one retried test -
   and confirm in the M00N Report dashboard: the run appears, tests show correct pass/fail/skip
   status, steps (if implemented) appear nested and in order, retries show as separate grouped
   attempts, and attachments are viewable.

---

## Self-Hosted Deployment

M00N Report can be self-hosted using Docker Compose or Kubernetes. Self-hosted is a separate plan, priced the same per user as Team.

**Docker Compose (quickstart):**

1. Get a subscription token at https://m00nreport.com/self-hosted
2. Download the Docker Compose bundle from that page (docker-compose.yml, env.example, nginx config)
3. Configure the required environment variables (subscription token, database URL, JWT secret, app base URL, object storage, SMTP)
4. Run:

```bash
docker compose up -d
```

**Kubernetes:** a Helm chart (values.yaml, production values, ArgoCD application) is available as a download from the same self-hosted page.

**MCP with self-hosted instance:**

```json
{
  "env": {
    "M00N_API_URL": "https://your-m00n-instance.company.com",
    "M00N_API_KEY": "m00n_mcp_your_key_here"
  }
}
```

---

## CI/CD Auto-Detection

The official reporters automatically detect CI/CD environments and capture metadata:

| Field | Recognized Variables |
|-------|---------------------|
| Pipeline | `pipeline`, `GITHUB_WORKFLOW`, `CI_PIPELINE_NAME`, `JOB_NAME` |
| Branch | `branch`, `git_branch`, `GITHUB_REF`, `CI_COMMIT_REF_NAME`, `BRANCH_NAME` |
| Commit | `commit`, `git_commit`, `GITHUB_SHA`, `CI_COMMIT_SHA`, `GIT_COMMIT` |
| Build Number | `build_number`, `GITHUB_RUN_NUMBER`, `CI_PIPELINE_ID`, `BUILD_NUMBER` |
| Build URL | `build_url`, `GITHUB_SERVER_URL`, `CI_PIPELINE_URL`, `BUILD_URL` |
| Environment | `environment`, `env`, `DEPLOY_ENV` |
| Trigger | `trigger`, `GITHUB_EVENT_NAME`, `CI_PIPELINE_SOURCE` |

---

## Key Features

- **Test case management:** Folder and suite repository, rich-text steps and expected results, attachments, tags, custom fields, per-project display IDs, case dependencies, bulk create/update/clone, CSV import, archive and restore
- **Manual test execution:** Executions with assignees, target environment, planned dates, per-step results and case snapshotting, plus reusable test collections
- **Agentic test execution:** An AI agent can run a manual execution over MCP, not just plan one - read each case's steps as data, exercise them with its own tooling, write pass/fail per step with issue notes, and let case, group and execution statuses derive themselves
- **Release management:** Aggregate manual execution progress and automated launch results into one release-readiness view, with optional Jira sprint linking
- **Coverage analysis:** Jira issue links on cases plus bidirectional autotest gap analysis, including autotests in recent launches that no test case owns
- **Project health score:** Composite 0-100 across automation stability, coverage and manual execution, with prioritized recommendations
- **AI and MCP:** 60 MCP tools (34 state-changing, 26 read-only, 10 flagged destructive), 7 prompts, 8 resources, over stdio or a hosted remote connector
- **Automated results:** Five official reporters plus a documented OpenAPI 3.0.3 ingest contract, with runs, tests and steps streamed live over WebSocket while the suite is still executing
- **Flaky test detection:** Automatic identification of flaky tests across runs
- **Slowest tests report:** Find performance bottlenecks in your test suite
- **CI triggering:** Start jobs on GitHub Actions, GitLab CI, Jenkins and TeamCity
- **Notifications:** Native Slack, Discord, Microsoft Teams, Telegram, Email integrations
- **Security:** PostgreSQL RLS tenant isolation, AES-256 encryption, SSO (OIDC/SAML) with group-to-project JIT mapping, four roles including a non-billable Guest, project-level API keys
- **Monitoring:** Prometheus metrics and Grafana dashboards included