[
https://issues.apache.org/jira/browse/OAK-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976097#comment-14976097
]
Tomek Rękawek commented on OAK-2106:
------------------------------------
{quote}Let's say the estimator measures a lag of 2 seconds at time T. That is,
secondaries have synced up to T-2s. At T+5s the secondaries still lag behind at
T-2s.{quote}
Let's have S - secondary optime, P - primery optime, T - current time. The lag
is measured as S-P, not S-T. It should allow to avoid the case in which the lag
is large, but we happen to measure it right after some operation has been
applied.
If we want to make it more reliable we can measure eg. 10 last values and
return the largest one.
{quote}I'm also a bit concerned about introducing a dependency from
MongoDocumentStore to classes like UnmergedBranches and UnsavedModifications.
I would rather like to see a solution where the client of the DocumentStore can
express how fresh the document needs to be when it reads from the store.{quote}
It concerns me as well (as this is some kind of circular dependency), but I
wasn't able to find something better. The access to unmerged branches is
necessary so we won't ask the secondary about the path belonging to branch. It
doesn't depend on the time, as user may modify many nodes (which'll result in
creating branch) and keep the changes unmerged for a very long time.
Situation looks a bit different with the UnsavedModifications, as they are
saved on a regular basis ({{asyncDelay}}) - we can add this value to the
estimated lag to be sure that background update thread has run and the changes
has been replicated.
{quote}I would rather like to see a solution where the client of the
DocumentStore can express how fresh the document needs to be when it reads from
the store. I think this also means the decision whether a read can be directed
to a secondary must not depend on the lag as a duration, but should rather
calculate a time when it is safe to read from a secondary.{quote}
We can take the {{find(maxCacheAge)}} parameter into consideration in the
{{getMongoReadPreference}}, however it doesn't solve the issue with the
unmerged branches.
{quote}The tricky part here is how to handle time differences on the machines
where the Oak cluster nodes are running and the MongoDB replica set. Each
change on a document is associated with a revision, where the timestamp of the
revision is tied to the local clock where the revision was created. The oplog
timestamp on the other hand is derived from the primary replica set member
clock, I assume.{quote}
The replication set status is taken from the primary. For each secondary member
we have 3 times available:
* optime - secondary time of the last operation applied,
* lastHeartbeat - secondary time of the last heartbeat sent,
* lastHeartbeatRecv - primary time of the last heartbeat received.
Primary member provides:
* optime,
* current timestamp.
As stated above, I estimate lag by subtracting primary optime from secondary
optime. These two times comes from different machines and therefore clock
differences will make it less accurate.
The other way of measuring the lag would be comparing lastHeartBeatRecv and
current time stamp. These two times comes from the same machine (primary). It
tells us how often the secondary ask for changes, but not how long does it take
to apply them. Maybe the first thing is more important - if so, I can change
the estimation method.
> Optimize reads from secondaries
> -------------------------------
>
> Key: OAK-2106
> URL: https://issues.apache.org/jira/browse/OAK-2106
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, mongomk
> Reporter: Marcel Reutegger
> Assignee: Marcel Reutegger
> Labels: performance, scalability
>
> OAK-1645 introduced support for reads from secondaries under certain
> conditions. The current implementation checks the _lastRev on a potentially
> cached parent document and reads from a secondary if it has not been
> modified in the last 6 hours. This timespan is somewhat arbitrary but
> reflects the assumption that the replication lag of a secondary shouldn't
> be more than 6 hours.
> This logic should be optimized to take the actual replication lag into
> account. MongoDB provides information about the replication lag with
> the command rs.status().
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
