Oooh, nice catch! That patch looks great. Thanks Simon! Mike McCandless
http://blog.mikemccandless.com On Wed, Jan 18, 2012 at 4:39 AM, Simon Willnauer <[email protected]> wrote: > this guy makes me sick. I think I found the issue, this seed > reproduces for me 9 out of 10 times > ant test -Dtestcase=SignatureUpdateProcessorFactoryTest > -Dtests.seed=-2302391d28a88bf9:0:-2b0b10982c64857e > -Dtests.multiplier=5 -Dargs="-Dfile.encoding=UTF-8" > > it turns out that this is caused by a fully deleted segment which > files are never removed. But since solr 3x closes the IW each time > after commit the subsequent IW tries to get rid of those files but > since they are now in a different directory impl. our MockDirWrapper > still has them in its unsyncedFiles set and once we close it throws an > exception. > > Actually I think we should try to delete those files immediately if we > drop a segment in commitMerge - this patch solves the problem for me: > > Index: lucene/src/java/org/apache/lucene/index/IndexWriter.java > =================================================================== > --- lucene/src/java/org/apache/lucene/index/IndexWriter.java (revision > 1232359) > +++ lucene/src/java/org/apache/lucene/index/IndexWriter.java (working copy) > @@ -3808,6 +3808,8 @@ > > if (dropSegment) { > readerPool.drop(merge.info); > + deleter.deleteNewFiles(merge.info.files()); > + assert !segmentInfos.contains(merge.info); > } > > if (infoStream != null) { > > 4.0 does the same thing so I think we should fix it there too > > > simon > > On Tue, Jan 17, 2012 at 2:15 PM, Apache Jenkins Server > <[email protected]> wrote: >> Build: https://builds.apache.org/job/Lucene-Solr-Maven-3.x/365/ >> >> 2 tests failed. >> FAILED: >> org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest.org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest >> >> Error Message: >> _c.tii >> >> Stack Trace: >> java.io.FileNotFoundException: _c.tii >> at >> org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:284) >> at >> org.apache.lucene.store.MockDirectoryWrapper.fileLength(MockDirectoryWrapper.java:716) >> at >> org.apache.lucene.store.MockDirectoryWrapper.crash(MockDirectoryWrapper.java:224) >> at >> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:559) >> at >> org.apache.solr.SolrTestCaseJ4.closeDirectories(SolrTestCaseJ4.java:82) >> at org.apache.solr.SolrTestCaseJ4.deleteCore(SolrTestCaseJ4.java:290) >> at >> org.apache.solr.SolrTestCaseJ4.afterClassSolrTestCase(SolrTestCaseJ4.java:72) >> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:616) >> at >> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) >> at >> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) >> at >> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) >> at >> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37) >> at org.junit.runners.ParentRunner.run(ParentRunner.java:236) >> at >> org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53) >> at >> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123) >> at >> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:616) >> at >> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164) >> at >> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110) >> at >> org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175) >> at >> org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:107) >> at >> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68) >> >> >> FAILED: >> org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest.org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest >> >> Error Message: >> java.lang.AssertionError: directory of test was not closed, opened from: >> org.apache.solr.core.MockDirectoryFactory.open(MockDirectoryFactory.java:34) >> >> Stack Trace: >> java.lang.RuntimeException: java.lang.AssertionError: directory of test was >> not closed, opened from: >> org.apache.solr.core.MockDirectoryFactory.open(MockDirectoryFactory.java:34) >> at >> org.apache.lucene.util.LuceneTestCase.afterClassLuceneTestCaseJ4(LuceneTestCase.java:310) >> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:616) >> at >> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) >> at >> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) >> at >> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) >> at >> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37) >> at org.junit.runners.ParentRunner.run(ParentRunner.java:236) >> at >> org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53) >> at >> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123) >> at >> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:616) >> at >> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164) >> at >> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110) >> at >> org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175) >> at >> org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:107) >> at >> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68) >> Caused by: java.lang.AssertionError: directory of test was not closed, >> opened from: >> org.apache.solr.core.MockDirectoryFactory.open(MockDirectoryFactory.java:34) >> at org.junit.Assert.fail(Assert.java:91) >> at >> org.apache.lucene.util.LuceneTestCase.checkResourcesAfterClass(LuceneTestCase.java:349) >> at >> org.apache.lucene.util.LuceneTestCase.afterClassLuceneTestCaseJ4(LuceneTestCase.java:278) >> ... 20 more >> >> >> >> >> Build Log (for compile errors): >> [...truncated 29070 lines...] >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
