System
Architecture
PostgreSQL is the system of record. Redis holds rate-limit and circuit-breaker state; SQS carries delivery work.
The event path
- Your application
POST /events with a tenant ID, type and payload.
- Ingestion API
Validate the request, check the tenant and apply its ingestion budget.
- PostgreSQL
Commit the event and its outbox entry in one transaction.
- Outbox poller
Publish pending entries to SQS, then mark confirmed sends published.
- SQS
Buffer the envelope or its S3 payload pointer.
- Delivery worker
Apply delivery controls, resolve the payload and sign the HTTP POST.
- Destination
The worker’s RECEIVER_URL receives the signed envelope.
One active publisher
The dedicated poller uses pg_try_advisory_lock(728401) on a single database connection. The lock elects a publisher; standby pollers wait. The API does not publish directly to SQS.
A failed send leaves the entry pending. A crash after SQS accepts the send but before the published flag is stored can cause duplicate publication.
Payloads and failure storage
S3 stores oversized payloads and dead-letter archives. Redis is not the queue. On retry, the worker changes visibility on the same SQS message. On exhaustion it archives and persists failure state before acknowledgement.
Delivery semantics
Delivery is at-least-once with bounded retries, not exactly-once. A permanently failing receiver is not guaranteed eventual delivery. Primary writes and replica reads are separate; a healthy replica may temporarily show older state.