MongoDB oplog

The MongoDB oplog (operations log) is a special capped collection, local.oplog.rs, that records every write to a replica set in order. Secondaries copy and apply these entries to stay in sync with the primary. Backup tools and change streams read the same log, which makes it the basis for point-in-time recovery.

How it works

Each oplog entry describes one idempotent operation, such as an insert, update or delete, with a timestamp. Because the collection is capped, old entries are overwritten once it is full. The time span it covers is the oplog window. For example, a 50 GB oplog on a cluster writing 2 GB of oplog per hour holds about 25 hours of history. A backup that combines a full mongodump with saved oplog slices can be replayed with mongorestore --oplogReplay to a chosen moment.

Why it matters in production

If a secondary falls behind by more than the oplog window, it cannot catch up and needs a full resync. If you back up oplog slices less often than the window, you lose the ability to restore between full backups. Size the oplog for your peak write rate, and watch replication lag against the window.

See also

• Change data capture

• RTO and RPO

• Schema migration

On the One2N blog

• Optimizing MongoDB backup strategy: lessons from achieving a 1-hour RPO

• MongoDB oplog to SQL parser

• MongoDB oplog to SQL parser exercise

References

1. MongoDB manual: replica set oplog