Why not Kafka, RabbitMQ or Redis/Bull

Status: released in v0.1.76 (2026-09-27) — design and as-built record in docs/design/event-bus-outbox-webhooks.md in the StandMeet repo.

The queue lives in the Postgres we already run, and no new service is added. River is one existing way to build a queue on Postgres; a self-built minimal queue on Postgres keeps the stack just as small. Four reasons rule out a broker.

1. A broker does not remove the outbox

The domain change and "the event happened" must commit together. Kafka, RabbitMQ and Redis cannot join a Postgres transaction. Using them still needs an outbox plus a relay that publishes to the broker, which only adds a hop. A queue in the same database makes enqueueing part of the transaction itself.

flowchart LR
  subgraph broker["With a broker: still an outbox, plus one hop"]
    direction LR
    a1[domain write] -->|same tx| b1[(outbox)]
    b1 --> c1[relay] -->|cross-process network| d1{{Kafka / RabbitMQ}}
    d1 --> e1[consumer]
  end
  subgraph pg["Queue on Postgres: enqueue inside the tx"]
    direction LR
    a2[domain write] -->|same tx| b2[(outbox)]
    b2 --> c2[relay] -->|same database, same tx| d2[(job table)]
    d2 --> e2[consumer]
  end

2. Wrong semantics

We need a job queue: each delivery retried on its own, on a backoff schedule, with an endpoint disabled after sustained failure.

• Kafka is an ordered log. It has no per-message delayed retry, so it needs hand-built retry and dead-letter topics, and one failing message blocks its partition.
• RabbitMQ can do delayed retry, but only by composing dead-letter exchanges and TTLs, or with a plugin.

3. Our Redis is built to lose data

docker-compose.prod.yml caps Redis at 256 MB with allkeys-lru: when full, it evicts the oldest keys. Sessions and rate-limit buckets survive eviction because they can be rebuilt; queued jobs would be dropped silently. The repo has no Bull or BullMQ, and the backend is Go, so Bull would also add a Node process.

4. Volume and cost

The event volume is hundreds to a few thousand a day: corpus edits, access requests, bookings. Postgres handles thousands of such writes per second. Self-hosting on a 1 GB box is part of the product.

Comparison

Option	New service	Typical memory	Transactional	Per-message delayed retry	Operations
Kafka	broker (KRaft still runs a JVM)	1 GB and up	dual write, needs outbox	no; needs retry topics	partitions, retention, disks, upgrades
RabbitMQ	Erlang server	150 MB and up	dual write, needs outbox	composed from DLX + TTL	queues, exchanges, durability config
Redis + Bull	none, but eviction and persistence must change	shared	dual write, needs outbox	yes	an extra Node process
River	none (a few tables in the existing Postgres)	near zero	same transaction	yes	backed up and migrated with the database; +1 Go module
Self-built on Postgres	none	near zero	same transaction	we implement it	a few hundred lines we own; the builder's SKIP LOCKED lease is a start

Storage growth does not vanish with another choice either: a Redis queue grows the same way and our Redis evicts silently when full; Kafka has built-in retention by time or size, at the cost of running another service. See storage-bounds.

When Kafka would make sense

A large multi-tenant SaaS, with many independent services consuming and replaying the same stream. Only the job runtime behind Jobs, Inspector and Runtime, and the relay inside internal/infra/events, would change; the domains would not notice (queue-behind-ports).

A further option to shrink the stack

Move sessions and rate limits into Postgres (UNLOGGED tables), and Redis can be removed entirely. This is a separate proposal.