[
https://issues.apache.org/jira/browse/OAK-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184835#comment-15184835
]
Chetan Mehrotra commented on OAK-4099:
--------------------------------------
Based on discussion with [~alex.parvulescu] have a following theory -
Compaction logic ensures that segment references less than 24 hr old are
remapped. So only possible way that above issue can occur is when any already
opened index did not saw any update for past 1 day and in that case
OakDirectory instance would continue to hold on to the segment references.
So one possible approach to avoid such a situation is to some how detect that
compaction has happened and close all open indexes and reopen them. Now would
need to see how we can detect from within oak-lucene that compaction has
occurred.
*Approach A* - One possible approach can be that when observers are called post
compaction we convey compaction as a cause as part of commit info. However
looking at code the observor are called with null commit info.
*Approach B* - Another approach can be that within oak-lucene we register a
{{GCMonitorTracker}} and in {{compacted}} callback do the refresh. This would
require oak-lucene to be aware of segment store
[~alex.parvulescu] [~mduerig] Thoughts on above? As per current impl I think #B
looks the way to go
> Lucene index appear to be corrupted with compaction enabled
> -----------------------------------------------------------
>
> Key: OAK-4099
> URL: https://issues.apache.org/jira/browse/OAK-4099
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
> Reporter: Chetan Mehrotra
> Assignee: Chetan Mehrotra
> Priority: Minor
> Fix For: 1.6
>
>
> While running on SegmentNodStore and online compaction enabled it can happen
> that access to Lucene index start failing with SegmentNotFoundException
> {noformat}
> Caused by:
> org.apache.jackrabbit.oak.plugins.segment.SegmentNotFoundException: Segment
> a949519a-8903-44f9-a17e-b6d83fb32186 not found
> at
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore.readSegment(FileStore.java:870)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.getSegment(SegmentTracker.java:136)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentId.getSegment(SegmentId.java:108)
> at
> org.apache.jackrabbit.oak.plugins.segment.Record.getSegment(Record.java:82)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.getNewStream(SegmentBlob.java:64)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexFile.loadBlob(OakDirectory.java:259)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexFile.readBytes(OakDirectory.java:307)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexInput.readBytes(OakDirectory.java:404)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexInput.readByte(OakDirectory.java:411)
> at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
> at
> org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadBlock(BlockTreeTermsReader.java:2397)
> at
> org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.seekCeil(BlockTreeTermsReader.java:1973)
> at
> org.apache.lucene.index.FilteredTermsEnum.next(FilteredTermsEnum.java:225)
> at
> org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:78)
> at
> org.apache.lucene.search.ConstantScoreAutoRewrite.rewrite(ConstantScoreAutoRewrite.java:95)
> at
> org.apache.lucene.search.MultiTermQuery$ConstantScoreAutoRewrite.rewrite(MultiTermQuery.java:220)
> at
> org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:288)
> at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:418)
> at
> org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:636)
> at
> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:683)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:378)
> {noformat}
> The above segmentId was mentioned in the compaction log
> {noformat}
> 06.03.2016 02:03:30.706 *INFO* [TarMK flush thread
> [/app/repository/segmentstore], active since Sun Mar 06 02:03:29 GMT 2016,
> previous max duration 8218ms]
> org.apache.jackrabbit.oak.plugins.segment.file.TarReader-GC Cleaned segments
> from data00233a.tar:
> 37ec786e-a9f7-46eb-a3b5-ce5d4777ea01,
> f36051fe-d8c4-46d1-ac1d-081946389eb6, fae91ff2-8ca6-4ac1-a8d8-d4bd09b7f6a6,
> 16d87f09-721b-4155-a9c8-b8ecf471bfc3,
> e641f1a3-b323-44e6-aad0-7b894a1efb69,
> edc9d141-6c05-42c9-a2a2-d7130fd9c826, b602372c-b17a-448a-a8e9-8bdccc64fb82,
> acc2f032-07ba-46ed-a9c7-d3a05ab53d7a,
> a7323ed2-b2de-4006-ae51-e4f84165a0e4,
> cb320c70-5ca9-4ed1-a972-e87a6bba9f9b, f45afd7e-5417-42dd-a2f7-4624f74b6c6e,
> c66f66ef-cdd0-4327-abc6-bf910cb5768d,
> 7f925a07-ff56-4613-ac8f-272a0e481926,
> 4ad044ec-3b2d-4c3e-aeb0-d5f5a04bc23e, 82f1c3aa-2e0c-421c-a033-e4ffcb6002c7,
> 1387655b-f633-4011-a55c-d9580e40929b,
> c50c94fc-2e8b-4904-a37f-0a33cc001312,
> 7915e9ce-bb9d-4628-ad6f-e7f2844b2399, e7cd013b-a147-426a-af29-fa025058a08a,
> f16d43b0-2113-4808-aea6-5910102e5c7d,
> ...
> *31edad2e-e14b-463d-a6af-540bac6009f1*,
> ...,
> *a949519a-8903-44f9-a17e-b6d83fb32186*,
> ...
> {noformat}
> *Note that system recovered after a restart so the corruption was transient*
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
