I think the issue is this particular codec (4.0 uncompressed stored fields writer) might do i/o in places like startDocument() / finishDocument() where the others dont.
I don't think testing exception handling this way is very effective, we should test the api instead. I opened an issue: https://issues.apache.org/jira/browse/LUCENE-5635 On Thu, May 1, 2014 at 5:24 AM, Robert Muir <[email protected]> wrote: > I did the refactoring (4.x: im porting forward to trunk now). > > Thing is it still doesnt fix the bug. but i think its easier to work with. > > On Wed, Apr 30, 2014 at 4:44 PM, Robert Muir <[email protected]> wrote: >> The exception handling in processDocument() is too complex. it must do >> try+finally with two separate cases: aborting exception and >> non-aborting exception, depending on where it happens. >> >> I think since the whole thing loops through fields, if we factor out >> processField it will be easier... >> >> On Wed, Apr 30, 2014 at 4:21 PM, Robert Muir <[email protected]> wrote: >>> reproduces if you tack on -Dtests.dups=100 ... looking >>> >>> On Wed, Apr 30, 2014 at 4:04 PM, <[email protected]> wrote: >>>> Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/20243/ >>>> >>>> 1 tests failed. >>>> REGRESSION: >>>> org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads >>>> >>>> Error Message: >>>> fdx size mismatch: docCount is 12 but fdx file size is 133 >>>> file=MockIndexOutputWrapper(org.apache.lucene.store.RAMOutputStream@3afabf87); >>>> now aborting this merge to prevent index corruption >>>> >>>> Stack Trace: >>>> java.lang.RuntimeException: fdx size mismatch: docCount is 12 but fdx file >>>> size is 133 >>>> file=MockIndexOutputWrapper(org.apache.lucene.store.RAMOutputStream@3afabf87); >>>> now aborting this merge to prevent index corruption >>>> at >>>> __randomizedtesting.SeedInfo.seed([70486912DEA23EAA:FF309021F56884D9]:0) >>>> at >>>> org.apache.lucene.codecs.lucene40.Lucene40StoredFieldsWriter.finish(Lucene40StoredFieldsWriter.java:233) >>>> at >>>> org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:96) >>>> at >>>> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:415) >>>> at >>>> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:512) >>>> at >>>> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622) >>>> at >>>> org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3203) >>>> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3179) >>>> at >>>> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:995) >>>> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:939) >>>> at >>>> org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads(TestIndexWriterWithThreads.java:168) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>> at >>>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) >>>> at >>>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) >>>> at >>>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) >>>> at >>>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) >>>> at >>>> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) >>>> at >>>> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) >>>> at >>>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) >>>> at >>>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>>> at >>>> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) >>>> at >>>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) >>>> at >>>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>>> at >>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>>> at >>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) >>>> at >>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) >>>> at >>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) >>>> at >>>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) >>>> at >>>> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) >>>> at >>>> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) >>>> at >>>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) >>>> at >>>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) >>>> at >>>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) >>>> at >>>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>>> at >>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) >>>> at >>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) >>>> at >>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>>> at >>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>>> at >>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>>> at >>>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) >>>> at >>>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>>> at >>>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) >>>> at >>>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) >>>> at >>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>>> at >>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) >>>> at java.lang.Thread.run(Thread.java:724) >>>> >>>> >>>> >>>> >>>> Build Log: >>>> [...truncated 251 lines...] >>>> [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads >>>> [junit4] 2> NOTE: reproduce with: ant test >>>> -Dtestcase=TestIndexWriterWithThreads >>>> -Dtests.method=testImmediateDiskFullWithThreads >>>> -Dtests.seed=70486912DEA23EAA -Dtests.slow=true -Dtests.locale=ko >>>> -Dtests.timezone=Etc/GMT+9 -Dtests.file.encoding=UTF-8 >>>> [junit4] ERROR 0.72s J2 | >>>> TestIndexWriterWithThreads.testImmediateDiskFullWithThreads <<< >>>> [junit4] > Throwable #1: java.lang.RuntimeException: fdx size >>>> mismatch: docCount is 12 but fdx file size is 133 >>>> file=MockIndexOutputWrapper(org.apache.lucene.store.RAMOutputStream@3afabf87); >>>> now aborting this merge to prevent index corruption >>>> [junit4] > at >>>> __randomizedtesting.SeedInfo.seed([70486912DEA23EAA:FF309021F56884D9]:0) >>>> [junit4] > at >>>> org.apache.lucene.codecs.lucene40.Lucene40StoredFieldsWriter.finish(Lucene40StoredFieldsWriter.java:233) >>>> [junit4] > at >>>> org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:96) >>>> [junit4] > at >>>> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:415) >>>> [junit4] > at >>>> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:512) >>>> [junit4] > at >>>> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622) >>>> [junit4] > at >>>> org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3203) >>>> [junit4] > at >>>> org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3179) >>>> [junit4] > at >>>> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:995) >>>> [junit4] > at >>>> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:939) >>>> [junit4] > at >>>> org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads(TestIndexWriterWithThreads.java:168) >>>> [junit4] > at java.lang.Thread.run(Thread.java:724) >>>> [junit4] 2> NOTE: test params are: codec=Lucene40, >>>> sim=DefaultSimilarity, locale=ko, timezone=Etc/GMT+9 >>>> [junit4] 2> NOTE: Linux 3.2.0-26-generic amd64/Oracle Corporation >>>> 1.7.0_25 (64-bit)/cpus=8,threads=1,free=241749984,total=258539520 >>>> [junit4] 2> NOTE: All tests run in this JVM: [TestSloppyMath, >>>> TestRamUsageEstimatorOnWildAnimals, TestReusableStringReader, >>>> TestByteArrayDataInput, TestSentinelIntSet, TestIndexWriterWithThreads] >>>> [junit4] Completed on J2 in 14.03s, 12 tests, 1 error <<< FAILURES! >>>> >>>> [...truncated 1260 lines...] >>>> BUILD FAILED >>>> /var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/build.xml:49: >>>> The following error occurred while executing this line: >>>> /var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/common-build.xml:1297: >>>> The following error occurred while executing this line: >>>> /var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/common-build.xml:921: >>>> There were test failures: 400 suites, 3109 tests, 1 error, 97 ignored (87 >>>> assumptions) >>>> >>>> Total time: 5 minutes 42 seconds >>>> Build step 'Invoke Ant' marked build as failure >>>> Recording test results >>>> Email was triggered for: Failure >>>> Sending email for trigger: Failure >>>> >>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
