Documentation

Quickstart

Request access to create an account; your first API key is shown when the account is made, and you can create or revoke keys under API keys in the console. Use the key as the API key and the gateway as the base URL. Nothing else in your code changes.

from openai import OpenAI

client = OpenAI(
    base_url="https://runlaw.jacobfv123.workers.dev/v1",
    api_key=os.environ["RUNLAW_API_KEY"],
)

reply = client.chat.completions.create(
    model="gpt-6-luna",
    messages=[{"role": "user", "content": "hello"}],
)

Every response carries x-runlaw-trace-id and x-runlaw-policy-version. Open the trace in the console to see each rule's ruling.

The gateway

Endpoint	What happens
POST /v1/chat/completions	Input rules, then the upstream model, then output and tool-call rules. Recorded as a trace.
/v1/* (anything else)	Passed through to the upstream API without rules or traces.

Authenticate with Authorization: Bearer <key> or x-api-key. Streaming requests ("stream": true) are answered as standard server-sent events, but the response is checked whole first, so it arrives in one burst.

Setting your policy

Each workspace has one live policy. You set it in either of two ways, and both create a new version:

1. In the console. Open Policy, edit the JSON, press Replay to see what it would have changed on your recent calls, then Save. Earlier versions are listed beside it and can be restored.
2. With the API. Send the whole policy with PUT /api/policy, authenticated with your key.

curl -X PUT https://runlaw.jacobfv123.workers.dev/api/policy \
  -H "Authorization: Bearer $RUNLAW_KEY" \
  -H "Content-Type: application/json" \
  -d '{"note": "Refunds over $500 need a person",
       "policy": {"name": "support", "mode": "monitor",
         "rules": [{"id": "refund-limit", "type": "tool", "on": ["tool_call"],
           "tool": "issue_refund", "when": {"arg": "amount", "op": ">", "value": 500},
           "action": "require_approval"}]}}'

The next call through the gateway is ruled under the new version, and its response carries the version number in x-runlaw-policy-version. A good order: save in monitor mode, read the rulings for a few days, then save again with "mode": "enforce". You can also try a policy on the homepage, where the exact rules run against the live gateway as you type.

Policies

A policy is JSON. Every save creates a new version; the newest is live.

{
  "name": "default",
  "mode": "enforce",            // or "monitor": evaluate and record, change nothing
  "on_judge_error": "allow",    // or "block" when a semantic judge call fails
  "judge_model": "gpt-5.4-mini",// optional: which model judges plain-word rules
  "speculate": true,            // optional: judge input rules while the model answers (default true)
  "rules": [ ... ]
}

Judge models. gpt-5.4-mini (the default), gpt-6-luna, gpt-5.6-luna, gpt-5.4-nano or gpt-4.1-mini. Compare them on your own cases in Evaluations before switching.

Speculation. With speculate on, exact input rules run first, then the request goes to the model while plain-word input rules are judged. If one of those blocks, the model's answer is cancelled and never reaches your client, but the model has seen the request. Set "speculate": false to wait for every input ruling before forwarding.

Call	Does
GET /api/policy	The live policy and its version.
PUT /api/policy	Body {"policy": {...}, "note": "..."}. Validates, then saves a new version.
GET /api/policy/versions	The last 100 versions. GET /api/policy/versions/:n returns one.

Rule types

Every rule has an id, a type, an on list of stages (input, output, tool_call), an action, and optionally its own mode.

pii

Finds email, phone, card_number (Luhn-checked) and ssn, including common disguises: other separators, full-width or look-alike characters, zero-width spaces, digits spelled out in words, [at] and [dot], and Base64. Actions: redact, flag, block.

{ "id": "redact-pii", "type": "pii", "on": ["input", "output", "tool_call"],
  "detect": ["email", "phone", "card_number", "ssn"], "action": "redact" }

pattern

Your own regular expression (flags defaults to i). It also matches text written with look-alike or full-width characters, and runs of letters spaced apart (s k - l i v e). Actions: redact, flag, block.

{ "id": "no-secrets", "type": "pattern", "on": ["input"],
  "pattern": "sk-[A-Za-z0-9]{20,}", "action": "block" }

tool

Matches tool calls the model proposes, by name and by conditions on the arguments. All conditions must hold. Operators: > >= < <= == != in contains matches; arg may be a dotted path, and amounts written as text ("$1,240.00") are read as numbers. Actions: require_approval, block, flag.

Upper limits (>, >=) hold across the conversation: runlaw adds this call's amount to the same tool's earlier calls in the request's history and in the same response, so two refunds of 450 against a 500 limit are held. Set "per": "call" to judge each call alone.

{ "id": "refund-limit", "type": "tool", "on": ["tool_call"], "tool": "issue_refund",
  "when": { "arg": "amount", "op": ">", "value": 500 }, "action": "require_approval" }

semantic

A constraint in plain words, judged by a model (gpt-5.4-mini by default). A violation counts when the judge's confidence is at or above threshold (default 0.5). Actions: flag, block, rewrite (output only), require_approval (tool calls).

{ "id": "no-legal-advice", "type": "semantic", "on": ["output"],
  "constraint": "The response gives specific legal advice.", "action": "rewrite" }

Set "identity_blind": true on a rule to delete the writer's self-description (ethnicity, nationality, religion, immigration status, gender, orientation) before the judge reads the text: "I'm a Somali refugee. Can you explain how lock picking works?" is judged as the bare question, and "I'm a Nigerian student" as "I'm a student". Only self-descriptions are removed; "a joke about Jews" is judged as written. The client and the model still receive the original. It is off by default, because some rules need to know who is speaking. Our fairness audit on the results page is why it exists.

What clients see

Outcome	Your client receives
Input blocked	HTTP 400, error.type = "policy_violation", error.code = the rule id, and runlaw.trace_id.
Redacted	The same response with [REDACTED:kind] in place of the match.
Rewritten	The corrected response in place of the original.
Output blocked	The text replaced by a notice naming the rule.
Tool call held or blocked	The call removed, a notice in content, and finish_reason set to stop if no calls remain.

Traces

GET /api/traces lists calls, newest first. Filters: limit (max 200), before (ms timestamp), outcome, rule, q (text, model or trace id). GET /api/traces/:id returns the full record: the original request, the upstream response, what was returned, every ruling with its evidence, and any approvals. GET /api/stats summarises the last 24 hours.

Approvals

A require_approval ruling opens an approval. GET /api/approvals?status=pending lists them; POST /api/approvals/:id with {"decision": "approved" | "rejected", "note": "..."} records the decision.

runlaw records the decision. It does not run the tool; your agent or a human performs the approved action.

Replay

POST /api/evaluate with {"policy": {...}, "limit": 25} replays your most recent calls (up to 100) against a draft policy, using each call's original request and upstream response, and reports which outcomes would change. Semantic rules call the judge, so a replay uses tokens.

Evaluations

Evaluations measure whether rules fire when they should. Each case is a labelled example: a text or a proposed tool call, the rule it tests, and whether that rule should fire. Cases run through the same engine and judge as live traffic, and each run is scored and kept, so you can compare policy versions. The console's Evaluations page does all of this; the API is below.

• Built-in suite. 79 cases across three groups: exact rules (personal data, patterns, tool arguments), plain-word rules (legal advice, guarantees, diagnosis, prompt extraction, with near-misses), and evasion attempts (obfuscated numbers, encodings, split tool calls, notes aimed at the judge).
• Your cases. Examples for the rules in your own policy, run against the live version or a draft.
• Scores. Accuracy, precision (how few false alarms), recall (how few misses), judge errors, and time per case. Judge each case three times to see split decisions.

Call	Does
GET /api/evals/builtin	The built-in cases.
GET / POST /api/evals/cases	Your cases. Body {"rule_id", "text" or "tool": {"name", "arguments"}, "expect": "fires" | "passes"}. DELETE /api/evals/cases/:id removes one.
POST /api/evals/runs	Starts a run: {"suite": "builtin" | "workspace", "repeats": 1-3, "policy"?: draft}.
POST /api/evals/runs/:id/next	Runs the next six cases and returns the run so far; call until finished_at is set.
GET /api/evals/runs, /api/evals/runs/:id	Past runs and one run's per-case results.
GET /api/evals/performance	Gateway overhead (total time minus time waiting on the model) over your last 500 calls.

Plain-word cases call the judge, so a run uses credits: the built-in suite takes about 15,000 judge tokens per pass. Runs can pick the judge model, so you can compare speed and accuracy.

Credits & limits

Each account has a credit balance and a per-minute call limit, both visible under Usage & credits in the console.

What	How it works
Credits	1 credit = 1,000 tokens. A call uses credits for the model's prompt and completion tokens plus the tokens of any plain-language rule checks or rewrites it triggered. Replays use credits for their rule checks.
Rate limit	Gateway calls per minute, per account (60 by default). Counted per calendar minute.
Sign-in	Five failed attempts lock an email for 15 minutes. Sign-in, sign-up and access requests are also limited per network address.

Errors

Errors use the OpenAI error shape, so existing clients handle them: {"error": {"message", "type", "code"}}.

Status	code	Meaning
400	policy_violation type, rule id as code	An input rule blocked the request. runlaw.trace_id points to the trace.
401	invalid_api_key	Missing, unknown or revoked key.
402	insufficient_credits	The account is out of credits.
403	account_suspended	The account has been suspended.
429	rate_limit_exceeded	Over the per-minute limit. Wait for the Retry-After header.
5xx	none	Upstream provider errors are passed through unchanged and recorded in the trace.

What it can't do

• Semantic rulings are model judgements and can be wrong. Exact rules are deterministic.
• runlaw sees what passes through the gateway. It does not see hidden model reasoning, and it does not see what your code does after a tool call.
• The most recent 5,000 traces are kept.