[
https://issues.apache.org/jira/browse/OAK-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Saurabh updated OAK-3380:
-------------------------------
Description:
Following up on this (a relatively old) thread \[1], we should do pruning of
property index structure asynchronously. The thread was never concluded.. here
are a couple of ideas picked from the thread:
* Move pruning to an async thread
* Throttle pruning i.e. prune only once in a while
** I'm not sure how that would work though -- an unpruned part would remain as
is until another index happens on that path.
Once we can move pruning to some async thread (reducing concurrent updates),
OAK-2673 + OAK-2929 can take care of add-add conflicts.
----
h6. Why is this an issue despite merge retries taking care of it?
A couple of cases which have concurrent updates hitting merge conflicts in our
product (Adobe AEM):
* Some index are very volatile (in the sense that indexed property switches its
values very quickly) e.g. sling job status, AEM workflow status.
* Multiple threads take care of jobs. Although sling maintains a bucketed
structure for job storage to reduce conflicts... but inside index tree the
bucket structure, at times, gets pruned and needs to be created in the next job
status change
While retries do take care of these conflict a lot of times and even when they
don't, AEM workflows has it's own retry to work around. But, retrying, IMHO, is
just a waste of time -- more importantly in paths where application doesn't
really have a control.
h6. Would this add to cost of traversing index structure?
Yes, there'd be some left over paths in index structure between asynchronous
prunes. But, I think the cost of such wasted traversals would be covered up
with time saved in avoiding the concurrent update conflict.
----
(cc [~tmueller], [~mreutegg], [~alex.parvulescu], [~chetanm])
\[1]:
http://mail-archives.apache.org/mod_mbox/jackrabbit-oak-dev/201506.mbox/%3ccadichf66u2vh-hlrjunansytxfidj2mt3vktr4ybkngpzy9...@mail.gmail.com%3E
was:
Following up on this (a relatively old) thread \[1], we should do pruning of
property index structure asynchronously. The thread was never concluded, here
are a couple of ideas picked from the thread:
* Move pruning to an async thread
* Throttle pruning i.e. prune only once in a while
** I'm not sure how that would work though -- an unpruned part would remain as
is until another index happens on that path.
Once we can move pruning to some async thread (reducing concurrent updates),
OAK-2673 + OAK-2929 can take care of add-add conflicts.
(cc [~tmueller], [~mreutegg], [~alex.parvulescu], [~chetanm])
\[1]:
http://mail-archives.apache.org/mod_mbox/jackrabbit-oak-dev/201506.mbox/%3ccadichf66u2vh-hlrjunansytxfidj2mt3vktr4ybkngpzy9...@mail.gmail.com%3E
> Property index pruning should happen asynchronously
> ---------------------------------------------------
>
> Key: OAK-3380
> URL: https://issues.apache.org/jira/browse/OAK-3380
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
> Affects Versions: 1.3.5
> Reporter: Vikas Saurabh
> Priority: Minor
> Labels: resilience
>
> Following up on this (a relatively old) thread \[1], we should do pruning of
> property index structure asynchronously. The thread was never concluded..
> here are a couple of ideas picked from the thread:
> * Move pruning to an async thread
> * Throttle pruning i.e. prune only once in a while
> ** I'm not sure how that would work though -- an unpruned part would remain
> as is until another index happens on that path.
> Once we can move pruning to some async thread (reducing concurrent updates),
> OAK-2673 + OAK-2929 can take care of add-add conflicts.
> ----
> h6. Why is this an issue despite merge retries taking care of it?
> A couple of cases which have concurrent updates hitting merge conflicts in
> our product (Adobe AEM):
> * Some index are very volatile (in the sense that indexed property switches
> its values very quickly) e.g. sling job status, AEM workflow status.
> * Multiple threads take care of jobs. Although sling maintains a bucketed
> structure for job storage to reduce conflicts... but inside index tree the
> bucket structure, at times, gets pruned and needs to be created in the next
> job status change
> While retries do take care of these conflict a lot of times and even when
> they don't, AEM workflows has it's own retry to work around. But, retrying,
> IMHO, is just a waste of time -- more importantly in paths where application
> doesn't really have a control.
> h6. Would this add to cost of traversing index structure?
> Yes, there'd be some left over paths in index structure between asynchronous
> prunes. But, I think the cost of such wasted traversals would be covered up
> with time saved in avoiding the concurrent update conflict.
> ----
> (cc [~tmueller], [~mreutegg], [~alex.parvulescu], [~chetanm])
> \[1]:
> http://mail-archives.apache.org/mod_mbox/jackrabbit-oak-dev/201506.mbox/%3ccadichf66u2vh-hlrjunansytxfidj2mt3vktr4ybkngpzy9...@mail.gmail.com%3E
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
