[
https://issues.apache.org/jira/browse/OAK-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14563544#comment-14563544
]
Marcel Reutegger commented on OAK-2829:
---------------------------------------
I started a prototype and committed it to a branch on github for now:
https://github.com/mreutegg/jackrabbit-oak/tree/OAK-2829
There is a new collection names 'journal', which contains documents with paths
of modified nodes between two consecutive background update operations. Those
changes are pushed before the _lastRevs are updated up to the root document,
which makes those changes available to other cluster nodes. A background read
operation will then query the new 'journal' collection for documents that match
the revision range defined by the last known _lastRevs on the root document and
those seen with the new background read. The background read operation then
combines them all and pushes them to the diff cache. This will then avoid calls
to calculate what changed at a given path for these external changes.
The same test with two Oak instances running the observation benchmark as
described on May 6th now works without filling up the observation queue. The
queue size graph shows spikes, but they drop pretty quickly:
!graph-1.png|width=100%!
It is also interesting to see that the number of external events in the queue
never exceeds one.
There are still a lot of TODOs, listed in
[JournalEntry|https://github.com/mreutegg/jackrabbit-oak/blob/OAK-2829/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/JournalEntry.java#L45]
> Comparing node states for external changes is too slow
> ------------------------------------------------------
>
> Key: OAK-2829
> URL: https://issues.apache.org/jira/browse/OAK-2829
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, mongomk
> Reporter: Marcel Reutegger
> Assignee: Marcel Reutegger
> Priority: Blocker
> Fix For: 1.3.0, 1.2.3
>
> Attachments: CompareAgainstBaseStateTest.java, graph-1.png, graph.png
>
>
> Comparing node states for local changes has been improved already with
> OAK-2669. But in a clustered setup generating events for external changes
> cannot make use of the introduced cache and is therefore slower. This can
> result in a growing observation queue, eventually reaching the configured
> limit. See also OAK-2683.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
