Replica sets and the oplog

Member states, replication lag and the oplog window, inside the server monitor.

The section appears only where a replica set exists. On a standalone server or through a router the capability is ruled out from the topology before anything is sent — a heading saying "replica set" above a message saying there is no replica set is not information.

It is read every fifth sampler pass: membership, election cadence and oplog reach change on the scale of minutes, and an open panel should not add a round trip per second.

#Members

Each member is listed with its state and its relationship to the set.

No primary is a state, not a failure. For a few seconds during an election it is entirely normal, so the panel calls it an election and says what follows from it rather than drawing an error.

#Lag has a basis, not just a value

So the panel writes a sentence, not a number. Five cases are named:

An arbiter is not a secondary with zero lag. It holds no data and has no replication position; showing "0 s" would place it in the lag column as the best member rather than outside the column entirely.

A member the set cannot see reports the epoch. A replication position of 1970-01-01 is a present and meaningless field; taken literally it produces "56 years behind".

#Two different kinds of unreachable

"The set says this member is unavailable" and "TableCore cannot reach it" are two different facts, and they are marked separately.

The first is about replication. The second is usually a firewall, or a hostname that only resolves inside the cluster's own network — which is the single most common surprise when connecting to a replica set from outside. The client's view comes from the driver's topology description, and no view at all is unknown rather than unavailable.

#The oplog window

The oplog window is the length of time the oplog covers. It is the deadline on every operation that has to catch up: a secondary that falls behind the window cannot resume from the oplog and needs a full resync, and a change stream or a backup that relies on the oplog fails the same way.

A shrinking window is a warning about all of those. The oplog has not changed size; the write rate has gone up, so the same bytes now cover less time.

Reading it takes three reads, not one command: both ends of the oplog collection, plus the statistics of the capped collection holding it — because without the configured maximum, "3 GB of oplog" says nothing about how much room is left.

Two details that would otherwise produce nonsense: the oplog's timestamps are BSON timestamps (seconds plus a counter within the second), not dates — read as dates they give a moment in 1970 and a window fifty-six years long. And times are formatted in minutes, hours and days, because "93,600.00 s" is not a number anybody reads as "just over a day", which is the entire question this reading answers.

#Permissions

The oplog window degrades separately from the member list. The oplog lives in the local database and reading it is a different privilege from clusterMonitor — plenty of users have one without the other, so losing one does not take the other away.

#What to do about it

• Lag that is measured and growing: the secondary cannot keep up. Look at what is writing, and at the secondary's own metrics.
• A window shrinking towards your backup or resync duration: increase the oplog size before it becomes the reason a resync is needed rather than a choice.
• A member the set sees and you cannot: a network problem on your side, not a replication problem. This is normal when a set is configured with internal hostnames — see SSH tunnels for why connecting to a single member is the shape that works through a bastion.