Cookbook

Respond to an incident

Walk a drift alert from the moment it fires through to a signed post-mortem.

This recipe takes a real signal — a drift alert from the model registry — and walks it through the full incident lifecycle. It's a template you can adapt for complaints, regulator queries, or any other inbound signal.

Scenario

Your nightly drift job pulls GET /v1/firm/me/models/drift and finds a new alert-severity signal: gpt-4-2024-11-20's rejection rate has jumped from 6% (baseline, last 90 days) to 17% (current, last 30 days), a delta of +11pp. Sample size is 142.

Step 1 — Open an incident

bash
curl -X POST https://api.bedrockcompliance.co.uk/v1/firm/me/incidents \
  -H "X-Bedrock-Key: bk_live_..." \
  -d '{
    "kind": "DRIFT_ALERT",
    "severity": "ALERT",
    "summary": "gpt-4-2024-11-20 rejection rate +11pp",
    "evidence": { "driftReportId": "drift_01HX...", "signal": "rejectionRate" }
  }'

Step 2 — Triage

  • Identify the affected model: (provider, version)
  • Pull the timeline: GET /v1/firm/me/models/timeline?provider=openai&version=gpt-4-2024-11-20
  • Identify the affected jobs: every job in the current window with that model
  • Identify the customers: every clientReference on those jobs

Step 3 — Investigate

Look for a cause:

  • Did the provider push a new minor version?
  • Did your prompt change?
  • Did the input distribution change (new product, new customer segment)?
  • Are the rejections concentrated in one product or one adviser?

Step 4 — Remediate

If the model is the cause:

  1. Pin advisers to the previous version in your back-office.
  2. File an impact assessment for the new version.
  3. Run a back-test on the previous month's rejected cases against the old version.
  4. Reinstate the new version only after the impact assessment is signed off.

Step 5 — Resolve and post-mortem

bash
curl -X POST https://api.bedrockcompliance.co.uk/v1/firm/me/incidents/inc_01HX.../resolve \
  -H "X-Bedrock-Key: bk_live_..." \
  -d '{
    "summary": "Pinned to gpt-4-2024-09-15. New version under impact assessment.",
    "rootCause": "Provider pushed an undocumented minor with stricter risk-scoring."
  }'

The post-mortem is signed and stored as a certificate, addressable forever via verify.bedrockcompliance.co.uk/c/....

See also