Test plan for the event bus

Updated · View the entry on sijie.xyz ↗

Status: released in v0.1.76 (2026-09-27) — design and as-built record in docs/design/event-bus-outbox-webhooks.md in the StandMeet repo. The full-suite acceptance below passed on 2026-09-27.

Internal mechanism is proven by UTs on a real Postgres, external behaviour by black-box e2e, and every phase re-runs the affected existing specs before it may close. UTs use real Postgres and httptest endpoints; no mocked database.

flowchart TB
  subgraph ut["UT · real Postgres · seconds"]
    u1[Mechanism: Recorder · trigger · relay · registries · schema parity]
    u2[Delivery: retry classes · backoff · cooldown · signing · SSRF · scope · embed hook upsert]
    u3[Saturation: pool · disk full · 429 · Meili down · backlog · waiters · starvation · timeout]
    u4[Conformance suite: any Jobs / Inspector / Runtime implementation must pass]
  end
  subgraph e2e["New e2e · 11 specs · black-box · real services"]
    e1[P1 index via bus · bulk-import bound · Tasks panel · upgrade]
    e2[P2 webhooks · every event type · fault injection]
    e3[P3 embed update hook]
    e4[P4 durable side effects · build settled after restart]
  end
  subgraph reg["Regression"]
    r1[each phase: the affected existing specs, REPEAT=5]
    r2[end: the full suite, one pass]
  end
  gates[7 gates · each shown red once on a planted sample]
  ut --> e2e --> reg
  gates -.-> reg

UT: what exists

Suite Covers
River conformance (~27) the Jobs / Inspector / Runtime contract: lifecycle, attempts, snooze, discard, unique-by-args, inspection, wait; run against River now and against any future implementation unchanged
Events relay / trigger / recorder (~26) row claiming; interleaved commits lose nothing; batch size; poison marking; concurrent relays never fan out twice; a non-coalescing subscriber gets every event of a subject; every watched column emits and no-op updates do not; undeclared types error
Saturation suite pool exhausted; disk full with relay backoff; 429 cooldown; Meili down, then the backlog drains; backlog surge alert; waiter cap; queue starvation; job timeout (saturation-degrades-gracefully)
Registry-driven idempotency (cmd/server/wire) every registered subscriber gets the same event twice and has one effect; it fails for a subscriber it does not know how to drive
Event-type registry every type declared once, named noun-then-verb, with a description and a subject pattern, and not left at the zero Exposure by accident
Per-use-case event coupling each producing use case records its event in the same transaction as its write
Webhook signing, classification, scope the Standard Webhooks test vector; every status and transport error → retryable / snoozed / discarded; raw never leaves, published slice, code denials, embed scope
Embed hook upsert the embed's update_hook_url creates or updates its attached endpoint
Schema parity TestMigrationsAddNothingToASchemaSQLDatabase; TestSQLPathSegmentMatchesGo

The interleaved-commit case comes from message-loss-guarantees; the registry-driven idempotency test from no-bypass-by-structure.

New e2e: what exists

Phase Spec What it proves
P1 events-index-via-bus an entry created through MCP corpus.create is searchable at once; an update and a delete show at once, and the receipt says so; a write during a Meili outage returns a receipt, is indexed once Meili is back, and survives a backend restart in between
P1 events-bulk-import-bound imported notes are all searchable, with at most N + slack index jobs and none discarded
P1 tasks-panel, tasks-panel-more the 12 Tasks panel items in tasks-panel, through the real UI
P1 upgrade-events-outbox an instance born before the bus upgrades by restart: the migration and River's migrator run at boot, an existing note edited afterwards is indexed, periodic jobs run on River
P2 webhooks a signed event arrives and verifies; the published slice only (sentinel form); raw never leaves; 500 × 2 then success, received once; 429 with Retry-After spends no attempt; 410 discarded, then re-delivered; send_test; secret rotation; a disabled endpoint receives nothing; a private address is refused
P2 webhook-event-types each exposed type, produced by a real action, reaches a * endpoint
P2 events-fault-injection an endpoint slower than the timeout is retried, then delivered; a refused connection turns retryable and starts the cooldown; after a Postgres restart an edit still reaches the receiver and search
P3 embed-update-hook the embed form's hook attaches an endpoint and an edit arrives signed; scope is the code's (sentinel form); saving the same URL keeps the endpoint and shows no new secret; a delete carries the last known subject; cards carry updated_at
P4 events-side-effects-durable an access-request notification that fails once still arrives; approval goes sending → sent and is marked replied only after the mail; a permanently failing relay shows failed and is not marked replied; a notification queued before a restart is delivered after it; a booking made during a restart still notifies the owner
P4 events-build-settled a build that settles after a backend restart still reaches the open preview
  • The webhook sink is a set of routes on the external mock (mock-stack/job-board/webhook_sink.go), with /__mock/set_delay for slow receivers.
  • Existing specs changed only in when they look, never what they assert (completion-hooks). The norm-outward-toolset golden gained the tasks.*, events.*, webhooks.* and jobs.fetch_result tools.
  • Gates are not e2e and keep no self-test script: each of the 7 was shown red once on a planted sample in a scratch copy (no-bypass-by-structure).
  • Dropped from the plan: a scheduled job-source fetch (not built, by decision) and webhook debouncing (webhooks never coalesce).

Regression

  • Each phase: the affected existing specs (embed, corpus, access, booking, microsite, obsidian, jobs, monitor, mail, search, supplier, calendar, upgrade, …) with REPEAT=5 to rule out flakes.
  • Order: each component's UTs first; each acceptance e2e written first and shown red on the unchanged code (make test-asis); then the implementation.

Final acceptance: every test in this repo green (passed 2026-09-27)

Command Covers
make lint secrets, env, backend (every check-*.sh gate, including the 7 of this plan), no-mock, app, sdk, e2e, im-bridge, verify-items
make backend-test Go tests, including the new UTs
make test-unit frontend and SDK unit tests
make im-bridge-test im-bridge
make stack-test whole-stack tests
make dsh-plugin-test every block installed into a real dsh and run
make test-fresh the full e2e suite from an empty volume, proving schema.sql and the migrations agree
make eval-* evaluations on the real agent loop (blocks, owner-mcp, ghost, summary, …)

One pass, no rerun-to-green. All of the above must be green in one pass. A failure is read from its logs and its mechanism found; it is never re-run until green. Only then: merge to main, release, upgrade sijie.xyz, deploy standmeet.com, and run the P3 real-environment acceptance, recorded under docs/real-env-verification/. Phase status is in events-roadmap.