[
https://issues.apache.org/jira/browse/LUCENE-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907569#action_12907569
]
Michael McCandless commented on LUCENE-2637:
--------------------------------------------
For the record, the problem w/ the opto is that it pulls the "unique"
FileChannel from the RandomAccessFile, positions it, and does non-positional
reads. This is not safe, because SimpleFSDir also does non-positional reads.
So eg if a BG merge is running, invoking .copyBytes when merging the fields,
and at the same time another thread is eg loading a stored doc from the same
open file w/ SimpleFSDIr, they conflict because they are both relying on the
stateful position of RAF.
NIOFSDir always does positional reads so it doesn't hit this.
But I agree we should pull the opto from 3x & trunk, open new issue to re-add
it, only add it back to trunk once we have solid tests. EG the new
TestCopyBytes I added does not catch this thread-safety issue (but I think,
though I didn't test, should catch the original corruption issue w/ the opto).
And, yes, we must now notify 3x/trunk users, who uses SimpleFSDir (because they
are on windows or because they explicitly instantiated it) that they may have
latent index corruption and should run CheckIndex.
> FSDirectory.copyBytes isn't safe for SimpleFSDirectory
> ------------------------------------------------------
>
> Key: LUCENE-2637
> URL: https://issues.apache.org/jira/browse/LUCENE-2637
> Project: Lucene - Java
> Issue Type: Bug
> Components: Tests
> Affects Versions: 3.1, 4.0
> Reporter: Robert Muir
> Assignee: Robert Muir
> Attachments: LUCENE-2637.patch, LUCENE-2637.patch
>
>
> the copyBytes optimization from LUCENE-2574 is not safe for
> SimpleFSDirectory, but works fine for NIOFSDirectory.
> With SimpleFSDirectory, the copyBytes optimization causes index corruption.
> see
> http://www.lucidimagination.com/search/document/36d2dbfc691909d5/bug_triggered_by_testindexwriter_testrandomstoredfields
> for background
> here are my steps to reproduce (most of the time, at least on windows):
> {noformat}
> 1. edit line 87 of TestIndexWriter to plugin the seed:
> random = newRandom(3312389322103990899L);
> 2. edit line 5138 of TestIndexWriter to force SimpleFSDirectory:
> Directory dir = new SimpleFSDirectory(index);
> 3. run this command:
> ant clean test-core -Dtestcase=TestIndexWriter
> -Dtestmethod=testRandomStoredFields -Dtests.iter=10
> -Dtests.codec="MockVariableIntBlock(29)"
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
