Great! Thanks! - Mark
http://about.me/markrmiller On Feb 8, 2014, 12:43:01 PM, Michael McCandless <[email protected]> wrote: I committed a fix earlier today. This test calls IW.close(false), which aborts any running merges, and then in turns out, if you also had installed a merged segment warmer (this test does) IW would in fact delete files held open by that temporary SegmentReader. This is actually "harmless" (this SegmentReader will never be included in any NRT reader), but I fixed IW to close the reader first, and then delete the files. Mike McCandless http://blog.mikemccandless.com On Sat, Feb 8, 2014 at 10:56 AM, Mark Miller <[email protected]> wrote: > I don't see any non nrt reader. > > Are we sure you can't have files that are part of segments that get merged > away before an NRT reader happens to be opened on them to keep them around? > > It would be legal, but our check would be too aggressive. > > - Mark > > http://about.me/markrmiller > > On Feb 8, 2014, at 6:49 AM, Michael McCandless <[email protected]> > wrote: > >> I'll dig; related to LUCENE-5434. >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> >> On Sat, Feb 8, 2014 at 2:13 AM, <[email protected]> wrote: >>> Build: builds.flonkings.com/job/Lucene-4x-Linux-Java6-64-test-only/12233/ >>> >>> 1 tests failed. >>> REGRESSION: org.apache.lucene.index.TestNRTThreads.testNRTThreads >>> >>> Error Message: >>> Captured an uncaught exception in thread: Thread[id=122, name=Lucene Merge >>> Thread #1, state=RUNNABLE, group=TGRP-TestNRTThreads] >>> >>> Stack Trace: >>> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an >>> uncaught exception in thread: Thread[id=122, name=Lucene Merge Thread #1, >>> state=RUNNABLE, group=TGRP-TestNRTThreads] >>> at __randomizedtesting.SeedInfo.seed([C9D7583FDC434A55:520E4C249DB85C3E]:0) >>> Caused by: org.apache.lucene.index.MergePolicy$MergeException: >>> java.lang.AssertionError: MockDirectoryWrapper: file "_c.cfs" is still >>> open: cannot delete >>> at __randomizedtesting.SeedInfo.seed([C9D7583FDC434A55]:0) >>> at >>> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:545) >>> at >>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518) >>> Caused by: java.lang.AssertionError: MockDirectoryWrapper: file "_c.cfs" is >>> still open: cannot delete >>> at >>> org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:433) >>> at >>> org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:392) >>> at >>> org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:584) >>> at >>> org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:349) >>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3721) >>> at >>> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405) >>> at >>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482) >>> Caused by: java.lang.RuntimeException: unclosed IndexInput: _c.cfs >>> at >>> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:534) >>> at >>> org.apache.lucene.store.MockDirectoryWrapper$1.openSlice(MockDirectoryWrapper.java:953) >>> at >>> org.apache.lucene.store.CompoundFileDirectory.openInput(CompoundFileDirectory.java:271) >>> at >>> org.apache.lucene.codecs.BlockTreeTermsReader.<init>(BlockTreeTermsReader.java:121) >>> at >>> org.apache.lucene.codecs.appending.AppendingTermsReader.<init>(AppendingTermsReader.java:45) >>> at >>> org.apache.lucene.codecs.appending.AppendingPostingsFormat.fieldsProducer(AppendingPostingsFormat.java:51) >>> at >>> org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:116) >>> at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:95) >>> at >>> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:141) >>> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4236) >>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3706) >>> ... 2 more >>> >>> >>> >>> >>> Build Log: >>> [...truncated 791 lines...] >>> [junit4] Suite: org.apache.lucene.index.TestNRTThreads >>> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestNRTThreads >>> -Dtests.method=testNRTThreads -Dtests.seed=C9D7583FDC434A55 >>> -Dtests.slow=true -Dtests.locale=hu -Dtests.timezone=Asia/Ashgabat >>> -Dtests.file.encoding=UTF-8 >>> [junit4] ERROR 5.87s J4 | TestNRTThreads.testNRTThreads <<< >>> [junit4] > Throwable #1: >>> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an >>> uncaught exception in thread: Thread[id=122, name=Lucene Merge Thread #1, >>> state=RUNNABLE, group=TGRP-TestNRTThreads] >>> [junit4] > at >>> __randomizedtesting.SeedInfo.seed([C9D7583FDC434A55:520E4C249DB85C3E]:0) >>> [junit4] > Caused by: org.apache.lucene.index.MergePolicy$MergeException: >>> java.lang.AssertionError: MockDirectoryWrapper: file "_c.cfs" is still >>> open: cannot delete >>> [junit4] > at __randomizedtesting.SeedInfo.seed([C9D7583FDC434A55]:0) >>> [junit4] > at >>> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:545) >>> [junit4] > at >>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518) >>> [junit4] > Caused by: java.lang.AssertionError: MockDirectoryWrapper: file >>> "_c.cfs" is still open: cannot delete >>> [junit4] > at >>> org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:433) >>> [junit4] > at >>> org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:392) >>> [junit4] > at >>> org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:584) >>> [junit4] > at >>> org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:349) >>> [junit4] > at >>> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3721) >>> [junit4] > at >>> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405) >>> [junit4] > at >>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482) >>> [junit4] > Caused by: java.lang.RuntimeException: unclosed IndexInput: >>> _c.cfs >>> [junit4] > at >>> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:534) >>> [junit4] > at >>> org.apache.lucene.store.MockDirectoryWrapper$1.openSlice(MockDirectoryWrapper.java:953) >>> [junit4] > at >>> org.apache.lucene.store.CompoundFileDirectory.openInput(CompoundFileDirectory.java:271) >>> [junit4] > at >>> org.apache.lucene.codecs.BlockTreeTermsReader.<init>(BlockTreeTermsReader.java:121) >>> [junit4] > at >>> org.apache.lucene.codecs.appending.AppendingTermsReader.<init>(AppendingTermsReader.java:45) >>> [junit4] > at >>> org.apache.lucene.codecs.appending.AppendingPostingsFormat.fieldsProducer(AppendingPostingsFormat.java:51) >>> [junit4] > at >>> org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:116) >>> [junit4] > at >>> org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:95) >>> [junit4] > at >>> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:141) >>> [junit4] > at >>> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4236) >>> [junit4] > at >>> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3706) >>> [junit4] > ... 2 more >>> [junit4] 2> NOTE: test params are: codec=Appending, >>> sim=RandomSimilarityProvider(queryNorm=false,coord=no): {extra38=DFR >>> I(n)L1, docid=IB SPL-DZ(0.3), body=IB LL-L3(800.0), extra14=IB SPL-D2, >>> extra27=DFR GB2, extra39=DFR I(n)1, titleTokenized=IB LL-D1, extra24=DFR >>> I(n)3(800.0), packID=DFR I(ne)B2, extra30=DFR I(F)3(800.0), extra32=DFR >>> GB3(800.0), extra7=IB SPL-D2}, locale=hu, timezone=Asia/Ashgabat >>> [junit4] 2> NOTE: Linux 3.2.0-26-generic amd64/Sun Microsystems Inc. >>> 1.6.0_45 (64-bit)/cpus=8,threads=1,free=192248392,total=264765440 >>> [junit4] 2> NOTE: All tests run in this JVM: [TestDocValuesWithThreads, >>> TestTopDocsMerge, TestDocTermOrdsRewriteMethod, Test2BDocs, >>> TestVirtualMethod, TestParallelReaderEmptyIndex, TestDocumentWriter, >>> TestSegmentMerger, Nested1, Nested2, Nested3, Nested, TestPriorityQueue, >>> TestSloppyPhraseQuery, TestStoredFieldsFormat, TestTermVectorsFormat, >>> TestByteBlockPool, TestScoreCachingWrappingScorer, TestMaxTermFrequency, >>> TestMultiValuedNumericRangeQuery, TestPayloadNearQuery, TestMathUtil, >>> TestFieldCacheTermsFilter, TestImpersonation, TestToken, TestBitVector, >>> TestBytesStore, TestDeletionPolicy, TestNRTThreads] >>> [junit4] Completed on J4 in 6.02s, 1 test, 1 error <<< FAILURES! >>> >>> [...truncated 629 lines...] >>> BUILD FAILED >>> /var/lib/jenkins/workspace/Lucene-4x-Linux-Java6-64-test-only/checkout/lucene/build.xml:49: >>> The following error occurred while executing this line: >>> /var/lib/jenkins/workspace/Lucene-4x-Linux-Java6-64-test-only/checkout/lucene/common-build.xml:1308: >>> The following error occurred while executing this line: >>> /var/lib/jenkins/workspace/Lucene-4x-Linux-Java6-64-test-only/checkout/lucene/common-build.xml:941: >>> There were test failures: 381 suites, 2772 tests, 1 error, 78 ignored (65 >>> assumptions) >>> >>> Total time: 5 minutes 45 seconds >>> Build step 'Invoke Ant' marked build as failure >>> Recording test results >>> Email was triggered for: Failure >>> Sending email for trigger: Failure >>> >>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
