Services

Resources

Company

Debezium

Debezium is an open-source change data capture (CDC) platform. It reads a database's own transaction log, such as the PostgreSQL write-ahead log (WAL) or the MongoDB oplog, and turns every committed insert, update and delete into an event. It usually runs inside Kafka Connect, or standalone as Debezium Server.

How it works

On PostgreSQL, Debezium creates a logical replication slot and reads decoded changes from it. Each event carries the row before and after the change, the operation (c, u, d) and the log position, so consumers can process changes in order and resume after a restart.

For example, UPDATE orders SET status = 'paid' WHERE id = 42 becomes one event on the topic shop.public.orders. A search indexer, an email service and a warehouse loader can all read it, and the order service does not change.

Why it matters in production

The replication slot is the part that bites. Postgres keeps WAL until the slot confirms it has read it, so a stopped connector can fill the database disk. Settings on replicas matter too: in One2N's case a hot_standby_feedback setting let Postgres remove WAL that Debezium was still reading. Alert on slot lag and retained WAL size, not only on connector status.

See also

On the One2N blog

References

  1. Debezium documentation

  2. Debezium connector for PostgreSQL

Comic about Debezium: Postgres writes each order change to its write-ahead log and Debezium reads each one as it lands; setting order 42 to paid becomes one update event with before and after values on topic shop.public.orders; the search index, email and warehouse all read that event with no app code changed; a stuck reader lets WAL pile up to 92% disk, so the team alerts on slot lag.