Events & webhooks

The notable moments in a project land on one append-only event feed: run lifecycle (completed/failed/paused), tool calls, approvals, connections, budget and credit alerts, channel events. You consume it two ways: poll the feed, or register webhooks and receive the same envelope as signed POSTs. The live per-token deltas of a run ride its SSE stream only, not this feed.

The event envelope

Both delivery modes carry the identical shape:

{ "v": 1, "id": "evt_…", "type": "run.completed",
  "created_at": "2026-06-11T…Z", "tenant_id": "…",
  "smith_id": "smt_…",
  "actor": { "kind": "tenant", "id": "…", "token_id": "jti_…", "email": "…" },
  "data": { "stop_reason": "end_turn", "usage": { … } } }

actor is who acted, resolved from the token at the point of action: kind: "smith" (a smith token, or the smith itself on a scheduled or channel-driven turn), "tenant" (a tenant-admin token acting on a smith's behalf), or "operator" (Ingram staff, named by email, or by an opaque staff id when that token carries none — never by your own tenant id). token_id is the authorizing token's jti, empty when no token acted. Every event a run produces carries that run's actor, and the run record carries it too. It is null on events recorded before attribution shipped.

email names the human behind the action, when there is one: the signed-in console user, or the Ingram operator. It is empty for a machine caller — an API token you minted, or a smith acting for itself — and empty for autonomous work. Read it together with token_id: a console change carries an email with no token_id (its per-request token is never registered), an API change carries a token_id with no email, and platform-driven work carries neither.

Polling the feed

# Authorization: tenant-admin token (server-side only)
curl "https://api.cloud.ingram.tech/v1/events?type=run.completed&smith_id=smt_…&since=2026-06-10T00:00:00Z&limit=50"

The feed is the reliable source of truth: if a webhook delivery is ever missed, everything is still here. The console's Observe → Events renders this same feed with the exact payload per row.

Registering webhooks

# Authorization: tenant-admin token (server-side only)
curl https://api.cloud.ingram.tech/v1/tenant/webhooks \
  -H "Authorization: Bearer $IC_TOKEN" \
  -H "IC-Api-Version: 2026-05-01" \
  -H "Content-Type: application/json" \
  -d '{ "url": "https://you.example/ic/webhook",
        "events": ["run.completed", "run.failed", "approval.required"] }'
# → { "id": "whk_…", "secret": "whsec_…" }   ← shown ONCE; store it

• An empty events list subscribes to everything; otherwise a type matches exactly or by family prefix. Subscribing to slack catches every slack.* type.
• The url must be reachable on the public internet over https. Anything else — a non-https scheme, or a host that is or resolves to a private, loopback, or link-local address — is rejected with invalid_url (422), and the same check runs on every delivery attempt and every redirect hop.
• POST /v1/tenant/webhooks/{id}/test fires a sample event; PATCH edits the endpoint (url, events, or active) — { "active": false } pauses deliveries; DELETE removes it. POST …/{id}/rotate_secret rolls the signing secret without a verification gap (see Rotating the signing secret).
• GET /v1/tenant/webhooks/{id}/deliveries lists the endpoint's delivery attempts, and POST …/deliveries/{did}/redeliver re-sends one (see Delivery semantics).
• In the console, Settings → Webhooks lists your endpoints. Pick the events from the multiselect when adding one, and the row's ⋯ menu edits, tests, rotates the secret, pauses, or deletes it. Rotating reveals the new secret once and keeps the old one valid for the overlap window.

Verifying signatures

Every delivery carries X-IC-Event-Id (dedupe key) and X-IC-Signature: t=<unix ts>,v1=<hex>, where v1 = HMAC-SHA256(secret, "<t>." + raw_body). During a secret rotation the header carries one v1= per active secret — accept the delivery if any of them matches:

import hashlib, hmac

def verify(headers, raw_body: bytes, secret: str) -> bool:
    parts = headers["X-IC-Signature"].split(",")
    t = next(p[2:] for p in parts if p.startswith("t="))
    sigs = [p[3:] for p in parts if p.startswith("v1=")]
    expected = hmac.new(secret.encode(), f"{t}.".encode() + raw_body,
                        hashlib.sha256).hexdigest()
    return any(hmac.compare_digest(expected, s) for s in sigs)

Compute over the raw request body (before any JSON parsing), and reject timestamps older than a few minutes to block replays.

Rotating the signing secret

# Authorization: tenant-admin token (server-side only)
curl -X POST \
  https://api.cloud.ingram.tech/v1/tenant/webhooks/whk_123/rotate_secret \
  -H "Authorization: Bearer $IC_TOKEN" \
  -H "IC-Api-Version: 2026-05-01" \
  -H "Content-Type: application/json" \
  -d '{ "grace_seconds": 86400 }'
# → { "id": "whk_…", "secret": "whsec_…",           ← new secret, shown ONCE
#     "previous_secret_expires_at": "2026-…Z" }

Rotation mints a new signing secret and keeps the old one valid for an overlap window (grace_seconds, default 24h, max 7d). During it every delivery is signed with both secrets, so there's no gap: your endpoint keeps verifying against the old secret while you roll the new one out, then switches over before previous_secret_expires_at. Pass grace_seconds: 0 to cut over immediately with no overlap (any in-flight verifier on the old secret then fails). The new secret is shown once — store it like the one from create.

Tool calls run over MCP — Ingram Cloud is the MCP client and invokes your server directly — and each call's tool.executing / tool.completed markers land on this feed alongside the human-in-the-loop and onboarding moments (approval.required, connection.required, unbound_message). The envelope's top-level smith_id is which smith the event belongs to.

Delivery semantics

• Delivery is durable. Each event fans out to a persisted delivery record per subscribed webhook. A first attempt fires immediately; a miss (endpoint down, slow, or non-2xx) is retried with exponential backoff (roughly 1m → 5m → 15m → 1h → 3h) until it succeeds or the retry budget is spent, at which point the delivery is marked dead.

• The attempt history is yours to inspect. List a webhook's deliveries — status (pending, succeeded, dead), attempt count, last HTTP status, next-attempt time — and force a fresh attempt once your endpoint recovers:

  # Authorization: tenant-admin token (server-side only)
  curl https://api.cloud.ingram.tech/v1/tenant/webhooks/whk_123/deliveries \
    -H "Authorization: Bearer $IC_TOKEN" -H "IC-Api-Version: 2026-05-01"
  
  curl -X POST \
    https://api.cloud.ingram.tech/v1/tenant/webhooks/whk_123/deliveries/whd_456/redeliver \
    -H "Authorization: Bearer $IC_TOKEN" -H "IC-Api-Version: 2026-05-01"
  

• /v1/events is still the reconcile backstop. Retries make a dropped webhook recoverable, but for anything you can't afford to miss — approval.required, budget.threshold, run.completed — a periodic poll of GET /v1/events?since=<last-seen timestamp> (or cursor=) is the belt-and-braces path. Every event carries the same idempotent id on both surfaces.

• Use X-IC-Event-Id to dedupe; deliveries (including retries) are not guaranteed to arrive in order.

• Respond 2xx quickly (within 10s) and do real work async.

Event type catalog

These are the types delivered to the feed and webhooks. The pure run-stream frames (run.started, message.delta, message.completed, run.cancelled, run.restarted, reasoning) are documented with the run stream and land on the per-run timeline (GET …/runs/{rid}/events) only — not here. reasoning is deliberately among them: a model's reasoning summary belongs to the run, not on a webhook to a third-party endpoint.

Administrative events (the audit trail)

Administrative mutations land in the same feed, so "who changed this, with which token, and when" is answerable from GET /v1/events and deliverable to a webhook. Every one carries actor — see the event envelope — and none of them carry the credential they describe: a key, a client secret and a signing secret never appear in an event payload, only the fact that one was set, replaced or removed.

Subscribe to a family by prefix: events: ["agent."] selects every agent event, ["token."] every token event.

agent.updated is the draft changing; agent.version_published and agent.rollout_updated are what smiths actually run changing. Re-provisioning an existing (external_id, agent) pair returns the smith it already had and fires nothing — only a genuine creation emits smith.created.