[jira] [Commented] (OAK-3646) Inconsistent read of hierarchy
[ https://issues.apache.org/jira/browse/OAK-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268197#comment-15268197 ] Marcel Reutegger commented on OAK-3646: --- I'd rather not backport this change. The fix is quite extensive and is therefore not easy to backport. > Inconsistent read of hierarchy > --- > > Key: OAK-3646 > URL: https://issues.apache.org/jira/browse/OAK-3646 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.0, 1.2 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Labels: resilience > Fix For: 1.4, 1.3.14 > > > This is similar to OAK-3388, but about hierarchy information like which child > nodes exist at a given revision of the parent node. This issue only occurs in > a cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3646) Inconsistent read of hierarchy
[ https://issues.apache.org/jira/browse/OAK-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087346#comment-15087346 ] Chetan Mehrotra commented on OAK-3646: -- The big change finally arrived at the start of new year :) Quite a bit of stuff to go through! > Inconsistent read of hierarchy > --- > > Key: OAK-3646 > URL: https://issues.apache.org/jira/browse/OAK-3646 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.0, 1.2 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Labels: resilience > Fix For: 1.3.14 > > > This is similar to OAK-3388, but about hierarchy information like which child > nodes exist at a given revision of the parent node. This issue only occurs in > a cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3646) Inconsistent read of hierarchy
[ https://issues.apache.org/jira/browse/OAK-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044990#comment-15044990 ] Marcel Reutegger commented on OAK-3646: --- A work-in-progress is available in a github branch: https://github.com/mreutegg/jackrabbit-oak/tree/OAK-3646 > Inconsistent read of hierarchy > --- > > Key: OAK-3646 > URL: https://issues.apache.org/jira/browse/OAK-3646 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.0, 1.2 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Fix For: 1.4 > > > This is similar to OAK-3388, but about hierarchy information like which child > nodes exist at a given revision of the parent node. This issue only occurs in > a cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3646) Inconsistent read of hierarchy
[ https://issues.apache.org/jira/browse/OAK-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028698#comment-15028698 ] Marcel Reutegger commented on OAK-3646: --- The name RevisionVector is inspired by [Version Vectors|https://en.wikipedia.org/wiki/Version_vector]. A revision vector works very similar to a version vector. The third update rule is a bit different with revision vectors, because a revision is bound to the local clock. In Oak the third rule is implemented by OAK-3388 where the background read waits until the local clock is passed the most recent reported external _lastRev and the timestamp of the new head revision is therefore newer than the timestamps of the most recent visible external revisions. > Inconsistent read of hierarchy > --- > > Key: OAK-3646 > URL: https://issues.apache.org/jira/browse/OAK-3646 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.0, 1.2 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Fix For: 1.4 > > > This is similar to OAK-3388, but about hierarchy information like which child > nodes exist at a given revision of the parent node. This issue only occurs in > a cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3646) Inconsistent read of hierarchy
[ https://issues.apache.org/jira/browse/OAK-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028280#comment-15028280 ] Thomas Mueller commented on OAK-3646: - That sounds good to me. The RevisionComparator was my idea, and it was a bad idea. > It simply compares the revision with the revision vector to decide if the > revision is visible or not. Correct me if I'm wrong, the algorithm would be: for each revision in the document, get the upper bound revision from the RevisionVector, and compare it. Class name: what about RevisionHorizon? If it defines the line of what is visible. > Inconsistent read of hierarchy > --- > > Key: OAK-3646 > URL: https://issues.apache.org/jira/browse/OAK-3646 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.0, 1.2 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Fix For: 1.4 > > > This is similar to OAK-3388, but about hierarchy information like which child > nodes exist at a given revision of the parent node. This issue only occurs in > a cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3646) Inconsistent read of hierarchy
[ https://issues.apache.org/jira/browse/OAK-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026806#comment-15026806 ] Marcel Reutegger commented on OAK-3646: --- I started to work on a fix that gets rid of the current RevisionComparator. Its use is the primary source of trouble because it does not provide a stable comparison of revisions across cluster nodes. Ordering of revisions with different clusterIds may change when one of the revision timestamps becomes older than one hour. As a replacement for the RevisionComparator I'd like to introduce a RevisionVector. This is a replacement for Revision whenever the DocumentNodeStore reads a node state at a given 'revision'. Instead of relying on additional information from the RevisionComparator, the RevisionVector already contains the information about what Revisions from other cluster nodes are visible at that time (local revision). E.g. given a RevisionVector \[r7-0-1, r4-0-2] for the root node state on cluster node 1, this means this cluster nodes sees changes up to r4-0-2 from cluster node 2. Traversing down the tree will update the read revision vector according to _lastRev entries and explicit modifications on the document. Calculating a node state at some read revision vector does not need additional information to decide if a change is visible or not. It simply compares the revision with the revision vector to decide if the revision is visible or not. > Inconsistent read of hierarchy > --- > > Key: OAK-3646 > URL: https://issues.apache.org/jira/browse/OAK-3646 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.0, 1.2 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Fix For: 1.4 > > > This is similar to OAK-3388, but about hierarchy information like which child > nodes exist at a given revision of the parent node. This issue only occurs in > a cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3646) Inconsistent read of hierarchy
[ https://issues.apache.org/jira/browse/OAK-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010950#comment-15010950 ] Marcel Reutegger commented on OAK-3646: --- The issue happens when revisions of a change are purged from the RevisionComparator and the order of two revisions changes. With the revision comparator and delayed visibility of changes from other cluster nodes it frequently happens that a local revision L happens before a foreign revision F even though timestamp(F) < timestamp(L). The RevisionComparator maintains this visibility information for revisions no older than one hour. I think the only viable solution is to add this information to the 'readRevision' of a node state. Right now a node state is identified by a path and a revision. In addition the key must also include the most recent visible revision from other cluster nodes. This way the information of the RevisionComparator is captured in the key and will survive the RevisionComparator purge. > Inconsistent read of hierarchy > --- > > Key: OAK-3646 > URL: https://issues.apache.org/jira/browse/OAK-3646 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.0, 1.2 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Fix For: 1.4 > > > This is similar to OAK-3388, but about hierarchy information like which child > nodes exist at a given revision of the parent node. This issue only occurs in > a cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3646) Inconsistent read of hierarchy
[ https://issues.apache.org/jira/browse/OAK-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011126#comment-15011126 ] Marcel Reutegger commented on OAK-3646: --- Added another test similar to the previous one. This one creates child nodes and does not involve garbage collection: http://svn.apache.org/r1715007 > Inconsistent read of hierarchy > --- > > Key: OAK-3646 > URL: https://issues.apache.org/jira/browse/OAK-3646 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.0, 1.2 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Fix For: 1.4 > > > This is similar to OAK-3388, but about hierarchy information like which child > nodes exist at a given revision of the parent node. This issue only occurs in > a cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3646) Inconsistent read of hierarchy
[ https://issues.apache.org/jira/browse/OAK-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15008638#comment-15008638 ] Marcel Reutegger commented on OAK-3646: --- Added a test (currently ignored): http://svn.apache.org/r1714773 > Inconsistent read of hierarchy > --- > > Key: OAK-3646 > URL: https://issues.apache.org/jira/browse/OAK-3646 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.0, 1.2 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Fix For: 1.4 > > > This is similar to OAK-3388, but about hierarchy information like which child > nodes exist at a given revision of the parent node. This issue only occurs in > a cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
