[
https://issues.apache.org/jira/browse/LUCENE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless updated LUCENE-5931:
---------------------------------------
Attachment: LUCENE-5931.patch
OK, here's another iteration on the patch ...
bq. I have backported the fix to 4.6 and now I believe I am seeing another
serious issue here.
Thanks Vitaly, that's a good catch (opening to an older commit point
from an NRT reader that's carrying deletes in RAM), and we don't
properly handle that case because SegmentReader doesn't know whether
its liveDocs came from disk (matching the delGen from the
SegmentCommitInfo) or were carried in RAM from IndexWriter.
I fixed this in this new patch by recording whether the
SegmentReader is NRT, and then fixing the reopen logic to load
liveDocs from disk in that case. I folded in your example as another
test case (failed at first but now passes).
Also I moved the "best-effort detection of rm -rf index" up
higher... it's currently too low now, i.e. will fail to detect some
cases that it should.
> DirectoryReader.openIfChanged(oldReader, commit) incorrectly assumes given
> commit point has deletes/field updates
> -----------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-5931
> URL: https://issues.apache.org/jira/browse/LUCENE-5931
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
> Affects Versions: 4.6.1
> Reporter: Vitaly Funstein
> Assignee: Michael McCandless
> Priority: Critical
> Attachments: CommitReuseTest.java, LUCENE-5931.patch,
> LUCENE-5931.patch, LUCENE-5931.patch, LUCENE-5931.patch
>
>
> {{StandardDirectoryReader}} assumes that the segments from commit point have
> deletes, when they may not, yet the original SegmentReader for the segment
> that we are trying to reuse does. This is evident when running attached JUnit
> test case with asserts enabled (default):
> {noformat}
> java.lang.AssertionError
> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:188)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:326)
> at
> org.apache.lucene.index.StandardDirectoryReader$2.doBody(StandardDirectoryReader.java:320)
> at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:702)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromCommit(StandardDirectoryReader.java:315)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenNoWriter(StandardDirectoryReader.java:311)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:262)
> at
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:183)
> {noformat}
> or, if asserts are disabled then it falls through into NPE:
> {noformat}
> java.lang.NullPointerException
> at java.io.File.<init>(File.java:305)
> at
> org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:80)
> at
> org.apache.lucene.codecs.lucene40.BitVector.<init>(BitVector.java:327)
> at
> org.apache.lucene.codecs.lucene40.Lucene40LiveDocsFormat.readLiveDocs(Lucene40LiveDocsFormat.java:90)
> at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:131)
> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:194)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:326)
> at
> org.apache.lucene.index.StandardDirectoryReader$2.doBody(StandardDirectoryReader.java:320)
> at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:702)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromCommit(StandardDirectoryReader.java:315)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenNoWriter(StandardDirectoryReader.java:311)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:262)
> at
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:183)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
