Skip to main content
Skip to main content
SeaOtter
HomeSubmitBuildLive demoCriticsRubrics
Request access

BUILD WITH OTTERLOOP

Wire the critic into an agent in minutes.

OtterLoop is the agent-facing contract for SeaOtter's hostile critic. The same loop works on or off AgentOS, across any framework, model, and cloud: submit the work, read the verdict, revise, and iterate until the band clears your gate.

MCPHTTPPython SDKMultimodal artifactsLocalized feedback

INTEGRATION

Three copy-paste starts. One contract.

Everything routes to the same eval contract. The hosted API owns critic execution, conditioning, localization, rich returns, and the signed audit record. The MCP server and Python SDK are thin wrappers over that HTTP surface.

  • Localize `detail`, `rationale`, and `upgrades` by locale.
  • Anchor to spans, cells, slides, pages, frames, or timestamps.
  • Fetch rendered artifacts separately when the agent needs the media bytes.
  • Keep the canonical feedback bundle in JSON for fallback-safe automation.

INTEGRATION

Three ways in.

Everything routes to the same eval contract. The hosted API owns critic execution, conditioning, localization, rich returns, and the signed audit record. The MCP server and Python SDK are thin wrappers over that HTTP surface.

MCP

Use in Claude, Codex, Cursor, or any MCP-speaking runtime.

{ "mcpServers": { "otterloop": {
    "command": "python", "args": ["-m", "otterloop.mcp_server"],
    "env": { "OTTERLOOP_API_URL": "https://dev-api.seaotter.ai",
             "OTTERLOOP_API_KEY": "sk-otter-...",
             "OTTERLOOP_POLICY_ID": "acme-prod-acceptance" } } } }

curl

One-shot score call over HTTP.

curl -s https://dev-api.seaotter.ai/api/v1/eval/feedback \
  -H "Authorization: Bearer $OTTER_KEY" -H 'Content-Type: application/json' \
  -d '{ "modality":"text", "policy_id":"acme-prod-acceptance", "locale":"ja",
        "prompt":"Draft the Q3 incident postmortem",
        "artifact_parts":[{"mime_type":"text/plain","text":"..."}],
        "return_feedback_artifacts": true }'

Python SDK

Let the client drive produce → grade → revise until ship.

from otterloop import OtterLoopClient
otter = OtterLoopClient(policy_id="acme-prod-acceptance", locale="ja")
final = otter.loop(produce=lambda feedback: my_agent.revise(feedback), work=my_agent.first_draft(), modality="document", references=["file://brand-guide.pdf", "file://gold-postmortem.md"], max_rounds=5, target_band="ship")

DEVELOPER CONSOLE

Get your eval API key

Mint an eval-API key for your account, then copy a ready-to-paste MCP, Python SDK, or curl setup to wire any agent into SeaOtter's hostile critic. The secret is shown once — store it before you leave this page.

Loading keys…

AGENT QUICKSTART

Onboard an agent without a human in the loop after the first key.

SeaOtter is agent-native: an agent can discover this contract from /llms.txt, then run the whole loop over MCP or plain HTTP. The only human step today is minting the first eval key (above) — the agent does the rest. Every eval call carries Authorization: Bearer <sk-otter-...>. Bases: https://api.seaotter.ai (prod), https://dev-api.seaotter.ai (dev).

  1. 1. Get a key — A signed-in org user creates an account at /signup, then mints an eval key here (POST /api/v1/agent-keys, body {"name":"my-agent"}). The full sk-otter-<40 hex> secret is shown exactly once. Hand it to the agent as OTTERLOOP_API_KEY / the bearer token.
  2. 2. Connect (MCP or HTTP) — Drop the .mcp.json below into Claude / Codex / Cursor for read-only tools (otter_list_policies, otter_score, otter_iterate, otter_score_workflow, otter_get_feedback_artifact), or call the HTTP API directly. pip install "otterloop[mcp]" (or pip install otterloop for the stdlib SDK).
  3. 3. Score — POST /api/v1/eval/feedback with the artifact + the prompt the agent was given (+ optional policy_id, locale, references). Get back { run_id, verdict }. The verdict has score (0-100, lower = more flawed), band (ship / route_to_fix / quarantine / block), flaws[], upgrades[].
  4. 4. Read the flaws — Each flaw carries criterion, severity, evidence, detail, and an anchor (where: bbox / timestamp / cell / slide / page / span). upgrades[] are concrete fixes the agent can apply.
  5. 5. Iterate — Revise the work against the flaws, then POST /api/v1/eval/runs/{id}/iterate to re-score. Repeat until band clears your gate (e.g. ship). The Python SDK's otter.loop(produce=..., work=..., target_band="ship") drives this automatically.
  6. 6. Workflow / benchmark — POST /api/v1/eval/workflows/{id}/topology to score an end-to-end multi-step workflow graph and get a composite + per-step + chain critique. Discover policies (GET /api/v1/eval/policies) and rubrics (GET /api/v1/eval/rubrics) to condition grading.

Machine-readable: /llms.txt · OpenAPI spec · interactive API docs.

1 — mint a key (one-time, signed-in user)

Returns the full sk-otter-... secret exactly once.

curl -s -X POST https://api.seaotter.ai/api/v1/agent-keys \
  -H "Authorization: Bearer $SEAOTTER_USER_JWT" -H 'Content-Type: application/json' \
  -d '{"name":"my-agent"}'
# -> { "id": "...", "key": "sk-otter-...", "key_prefix": "sk-otter-abcde", ... }

2 — connect over MCP (.mcp.json)

Claude / Cursor. Codex uses [mcp_servers.otterloop] in config.toml.

{ "mcpServers": { "otterloop": {
    "command": "python", "args": ["-m", "otterloop.mcp_server"],
    "env": { "OTTERLOOP_API_URL": "https://api.seaotter.ai",
             "OTTERLOOP_API_KEY": "sk-otter-...",
             "OTTERLOOP_POLICY_ID": "acme-prod-acceptance" } } } }

3 — score over HTTP

One-shot grade -> verdict + run_id to keep iterating.

curl -s https://api.seaotter.ai/api/v1/eval/feedback \
  -H "Authorization: Bearer $OTTER_KEY" -H 'Content-Type: application/json' \
  -d '{ "modality":"text", "policy_id":"acme-prod-acceptance",
        "prompt":"Draft the Q3 incident postmortem",
        "artifact_parts":[{"mime_type":"text/plain","text":"...your work..."}],
        "return_feedback_artifacts": true }'
# -> { "run_id": "...", "verdict": { "score": ..., "band": "route_to_fix", "flaws": [...] } }

5 — iterate until it ships

Re-score a revision against the same run.

curl -s -X POST https://api.seaotter.ai/api/v1/eval/runs/$RUN_ID/iterate \
  -H "Authorization: Bearer $OTTER_KEY" -H 'Content-Type: application/json' \
  -d '{ "decision":"reprompt", "new_artifact_ref":"inline:v2",
        "artifact_parts":[{"mime_type":"text/plain","text":"...revised work..."}] }'

VERDICT CONTRACT

The agent acts on one schema.

The verdict is designed for frontier agents, not screenshots of human review. It carries score, band, flaws, upgrades, anchors, rationale, and rich-feedback artifact refs the agent can use directly.

Verdict schema

{
  "score": 0.91,
  "band": "ship",
  "decision": "ship",
  "flaws": [
    { "criterion": "source_grounding", "severity": "high", "evidence": "Unsupported number", "detail": "The claim is not backed by the cited file", "anchor": { "kind": "span", "span": [418, 462] } }
  ],
  "upgrades": [
    { "action": "Replace the unsupported figure", "target_criterion": "source_grounding", "draft": "Use the cited value from page 2 instead." }
  ],
  "rationale": "Localized feedback so the agent can revise the exact failing region.",
  "feedback_artifacts": [{ "kind": "annotated_png", "ref": "artifact://..." }]
}

CONDITIONING

The critic is conditional on your bar.

OtterLoop is not a generic "is this good" score. The contract can condition the verdict on your organisation's policy, the prompt or intent the agent was given, and the reference files it must obey.

Organisation policy

Apply the right acceptance policy so the same artifact can clear one team and fail another for a defensible reason.

Prompt and intent

Carry the original ask into the critic so it judges the work against the assignment, not against a generic idealized answer.

Reference files

Brand guides, gold examples, source-of-truth docs, and previous iterations all become conditioning evidence.

  • Anchors localize to bbox, point, span, cell, slide, page, or timestamp.
  • The band is a runtime policy decision, not model prose pretending to be a gate.
  • Rich returns let the same verdict drive both human review and machine revision.

MODALITIES

Multimodal in. Rich multimodal out.

The same loop covers text, code, images, decks, documents, spreadsheets, audio, video, and multi-step trajectories. Returns can include both the canonical verdict JSON and media a human or agent can read.

Try the live demoBrowse rubrics
MODALITIESRETURNS
Image or design frameAnnotated PNG plus flaw bounding boxes and a markdown report
Deck, PDF, or documentAnnotated pages, per-page notes, and machine-readable anchors
SpreadsheetFlagged cells, criterion-grounded notes, and structured deltas
Video or audioTimestamp markers, captions, and localized rationale
Text or codeSpan-anchored review with upgrade drafts the agent can apply
SeaOtterThe acceptance layer for enterprise agent work.
SubmitBuildLive demoCriticsRubrics

© 2026 SeaOtter. The acceptance layer for enterprise agent work.