[
https://issues.apache.org/jira/browse/LUCENE-9309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Willnauer resolved LUCENE-9309.
-------------------------------------
Fix Version/s: 8.6
master (9.0)
Lucene Fields: New,Patch Available (was: New)
Assignee: Simon Willnauer
Resolution: Fixed
> IW#addIndices(CodecReader) might delete files concurrently to IW#rollback
> -------------------------------------------------------------------------
>
> Key: LUCENE-9309
> URL: https://issues.apache.org/jira/browse/LUCENE-9309
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Major
> Fix For: master (9.0), 8.6
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> During work on LUCENE-9304 [~mikemccand] ran into a failure:
> {noformat}
> org.apache.lucene.index.TestAddIndexes > test suite's output saved to
> /home/mike/src/simon/lucene/core/build/test-results/test/outputs/OUTPUT-org.apache.lucene.index.TestAddIndexes.txt,
> copied below:
> > java.nio.file.NoSuchFileException:
> _gt_Lucene85FieldsIndex-doc_ids_6u.tmp
> > at
> __randomizedtesting.SeedInfo.seed([4760FA81FBD4B2CE:A147156E5F7BD9B0]:0)
> > at
> org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
> > at
> org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607)
> > at
> org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
> > at
> org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
> > at
> org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
> > at
> org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:449)
> > at
> org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2334)
> > at
> org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2275)
> > at
> org.apache.lucene.index.IndexWriter.rollback(IndexWriter.java:2268)
> > at
> org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:974)
> 2> NOTE: reproduce with: ant test -Dtestcase=TestAddIndexes
> -Dtests.method=testAddIndexesWithRollback -Dtests.seed=4760FA81FBD4B2CE
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=fr-GP -Dtests.t\
> imezone=Asia/Tbilisi -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> 2> NOTE: test params are: codec=Asserting(Lucene84):
> {c=PostingsFormat(name=LuceneFixedGap),
> id=PostingsFormat(name=LuceneFixedGap),
> f1=PostingsFormat(name=LuceneFixedGap), f2=BlockTreeOrds(blocksize=128)\
> , version=BlockTreeOrds(blocksize=128), content=FST50},
> docValues:{dv=DocValuesFormat(name=Lucene80),
> soft_delete=DocValuesFormat(name=Lucene80),
> doc=DocValuesFormat(name=Lucene80), id=DocValuesFormat(name=\
> Asserting), content=DocValuesFormat(name=Asserting),
> doc2d=DocValuesFormat(name=Lucene80)}, maxPointsInLeafNode=982,
> maxMBSortInHeap=5.837219998050092,
> sim=Asserting(org.apache.lucene.search.similarities.As\
> sertingSimilarity@6ce38471), locale=fr-GP, timezone=Asia/Tbilisi
> {noformat}
> While this unfortunately doesn't reproduce it's likely a bug that exists for
> quite some time but never showed up until LUCENE-9147 which uses a temporary
> output. That's fine but with IW#addIndices(CodecReader...) not registering
> the merge it does in the IW we never wait for the merge to finish while
> rollback and if that merge finishes concurrently it will also remove these
> .tmp files.
> There are many ways to fix this and I can work on a patch, but hey do we
> really need to be able to add indices while we index and do that on an open
> and live IW or can it be a tool on top of it?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
