[jira] [Commented] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[ https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510283#comment-15510283 ] Vikas Saurabh commented on OAK-4805: [~chetanm], I'd like to backport this to 1.0 as well, but in absence of custom analyzer feature in 1.0 I don't know how to test a mis-configured index. Ideas? > Misconfigured lucene index definition can render the whole system unusable > -- > > Key: OAK-4805 > URL: https://issues.apache.org/jira/browse/OAK-4805 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh > Labels: candidate_oak_1_0 > Fix For: 1.6, 1.4.8, 1.5.11, 1.2.20 > > Attachments: OAK-4805.patch > > > Mis-configured index definition can throw an exception while collecting > plans. This causes any query (even unrelated ones) to not work as cost > calculation logic would consult a badly constructed index def. Overall a > mis-configured index definition can practically grind the whole system to > halt as the whole query framework stops working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[ https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510194#comment-15510194 ] Vikas Saurabh commented on OAK-4805: I haven't got time to work on the issue so opened OAK-4836 for avoiding filling up logs. Would commit the patch soon (with additional change of IAE to Exception). > Misconfigured lucene index definition can render the whole system unusable > -- > > Key: OAK-4805 > URL: https://issues.apache.org/jira/browse/OAK-4805 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh > Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4 > Fix For: 1.6 > > Attachments: OAK-4805.patch > > > Mis-configured index definition can throw an exception while collecting > plans. This causes any query (even unrelated ones) to not work as cost > calculation logic would consult a badly constructed index def. Overall a > mis-configured index definition can practically grind the whole system to > halt as the whole query framework stops working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[ https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15492722#comment-15492722 ] Vikas Saurabh commented on OAK-4805: Ack. Would try to see how quickly can I do the complete one. Else, I'd commit this patch and open another issue as you suggested. > Misconfigured lucene index definition can render the whole system unusable > -- > > Key: OAK-4805 > URL: https://issues.apache.org/jira/browse/OAK-4805 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh > Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4 > Fix For: 1.6 > > Attachments: OAK-4805.patch > > > Mis-configured index definition can throw an exception while collecting > plans. This causes any query (even unrelated ones) to not work as cost > calculation logic would consult a badly constructed index def. Overall a > mis-configured index definition can practically grind the whole system to > halt as the whole query framework stops working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[
https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15492487#comment-15492487
]
Chetan Mehrotra commented on OAK-4805:
--
Current patch would address this specific issue but I would like to take it one
step further. Per current design we can have bad index in following ways
# Index got corrupted (for some reason) - In this case for each query
{{IndexTracker}} would try to acquire {{IndexNode}} which would fail while
getting constructed i.e. creation of IndexSearcher would lead to read of index
resulting in an exception. This index would then be ignored due to catch block
in {{IndexTracker#findIndexNode}}
# Index content is valid but some later change caused the IndexDefinition to
have invalid configuration - In this case IndexNode would get created but
exception from it would break down the query engine (current issue)
In both cases once we have a bad index then it would continue to participate in
subsequent query evaluation and
# flood the logs with failure message
# Slow down the processing as for Case #1 above IndexTracker would still try to
initialize it
So to address it properly we should do following
# Have some state in {{IndexTracker}} around failed index. If some failure is
seen either due to #1 and #2 then we mark such index paths as failed and log
the exception
# Upon lookup for index for path we check if its failed - If yes then ignore
# The state would should be cleared once any index gets updated i.e. in
{{IndexTracker#diffAndUpdate}}
As chance of index resuming back to normal is only after any index gets update
it should be fine to clear the state only upon index update. With this we
should see such warning every 5 sec at minimum. If require we can make such
logging more time bound
This would ensure that a fail index should not affect broader working of system.
Probably all this can be done in another issue and current issue gets fixed
with proposed patch as its much more critical. In that case would suggest to
catch {{Exception}} instead of {{IllegalArgumentException}} to guard against
other possible exceptions
> Misconfigured lucene index definition can render the whole system unusable
> --
>
> Key: OAK-4805
> URL: https://issues.apache.org/jira/browse/OAK-4805
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
> Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4
> Fix For: 1.6
>
> Attachments: OAK-4805.patch
>
>
> Mis-configured index definition can throw an exception while collecting
> plans. This causes any query (even unrelated ones) to not work as cost
> calculation logic would consult a badly constructed index def. Overall a
> mis-configured index definition can practically grind the whole system to
> halt as the whole query framework stops working.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
