[
https://issues.apache.org/jira/browse/OAK-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529581#comment-15529581
]
Ian Boston commented on OAK-3547:
---------------------------------
I have tested the patch in Sling and AEM 6.2+Oak 1.6, no functional regressions
or errors were seen.
There is an indication that opening the index is significantly slower with the
patch as, in order to verify all the files in the index the files are read and
a sha1 is generated from those files to ensure that the files are not damaged.
Instrumenting the OakDirectory constructor call, which in the generational
version validates the contents of the directory before opening, reveals.
WIthout OAK-3547 patch
{code}
t count max mean min stddev p50 p75 p95 p98
p99 p999 mean_rate m1_rate m5_rate m15_rate rate_unit
duration_unit
1475066389 40 10.221184 0.148839 0.022663
0.882145 0.072211 0.072211 0.078419 0.15831 0.15831
10.221184 0.07476 0.00111 0.227486 0.687796 calls/second
milliseconds
{code}
With OAK-3547 patch
{code}
t count max mean min stddev p50 p75 p95 p98
p99 p999 mean_rate m1_rate m5_rate m15_rate rate_unit
duration_unit
1475063657 40 571.811075 378.76475 0.125733
209.479519 492.429012 492.429012 492.429012 571.811075
571.811075 571.811075 0.082329 0.002645 0.269424
0.727275 calls/second milliseconds
{code}
How much the difference is will depend on the size of the index files. The
patch may also transfer the IO read operation on the index from outside the
OakDirectory constructor to inside the OakDirectory constructor, so these
readings may or may not be significant.
If they prove to be significant, then the SHA1 on files could be dropped on
every open directory open and some other check be performed. Other checks won't
be as robust as a full SHA1 check.
> Improve ability of the OakDirectory to recover from unexpected file errors
> --------------------------------------------------------------------------
>
> Key: OAK-3547
> URL: https://issues.apache.org/jira/browse/OAK-3547
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
> Affects Versions: 1.4
> Reporter: Ian Boston
> Fix For: 1.6
>
>
> Currently if the OakDirectory finds that a file is missing or in some way
> damaged, and exception is thrown which impacts all queries using that index,
> at times making the index unavailable. This improvement aims to make the
> OakDirectory recover to a previously ok state by storing which files were
> involved in previous states, and giving the code some way of checking if they
> are valid.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
