I did the refactoring (4.x: im porting forward to trunk now). Thing is it still doesnt fix the bug. but i think its easier to work with.
On Wed, Apr 30, 2014 at 4:44 PM, Robert Muir <[email protected]> wrote: > The exception handling in processDocument() is too complex. it must do > try+finally with two separate cases: aborting exception and > non-aborting exception, depending on where it happens. > > I think since the whole thing loops through fields, if we factor out > processField it will be easier... > > On Wed, Apr 30, 2014 at 4:21 PM, Robert Muir <[email protected]> wrote: >> reproduces if you tack on -Dtests.dups=100 ... looking >> >> On Wed, Apr 30, 2014 at 4:04 PM, <[email protected]> wrote: >>> Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/20243/ >>> >>> 1 tests failed. >>> REGRESSION: >>> org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads >>> >>> Error Message: >>> fdx size mismatch: docCount is 12 but fdx file size is 133 >>> file=MockIndexOutputWrapper(org.apache.lucene.store.RAMOutputStream@3afabf87); >>> now aborting this merge to prevent index corruption >>> >>> Stack Trace: >>> java.lang.RuntimeException: fdx size mismatch: docCount is 12 but fdx file >>> size is 133 >>> file=MockIndexOutputWrapper(org.apache.lucene.store.RAMOutputStream@3afabf87); >>> now aborting this merge to prevent index corruption >>> at >>> __randomizedtesting.SeedInfo.seed([70486912DEA23EAA:FF309021F56884D9]:0) >>> at >>> org.apache.lucene.codecs.lucene40.Lucene40StoredFieldsWriter.finish(Lucene40StoredFieldsWriter.java:233) >>> at >>> org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:96) >>> at >>> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:415) >>> at >>> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:512) >>> at >>> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622) >>> at >>> org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3203) >>> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3179) >>> at >>> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:995) >>> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:939) >>> at >>> org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads(TestIndexWriterWithThreads.java:168) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:606) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) >>> at >>> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) >>> at >>> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) >>> at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) >>> at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> at >>> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) >>> at >>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) >>> at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) >>> at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) >>> at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) >>> at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) >>> at >>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) >>> at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> at >>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) >>> at >>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) >>> at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> at >>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) >>> at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> at >>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) >>> at >>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) >>> at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) >>> at java.lang.Thread.run(Thread.java:724) >>> >>> >>> >>> >>> Build Log: >>> [...truncated 251 lines...] >>> [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads >>> [junit4] 2> NOTE: reproduce with: ant test >>> -Dtestcase=TestIndexWriterWithThreads >>> -Dtests.method=testImmediateDiskFullWithThreads >>> -Dtests.seed=70486912DEA23EAA -Dtests.slow=true -Dtests.locale=ko >>> -Dtests.timezone=Etc/GMT+9 -Dtests.file.encoding=UTF-8 >>> [junit4] ERROR 0.72s J2 | >>> TestIndexWriterWithThreads.testImmediateDiskFullWithThreads <<< >>> [junit4] > Throwable #1: java.lang.RuntimeException: fdx size >>> mismatch: docCount is 12 but fdx file size is 133 >>> file=MockIndexOutputWrapper(org.apache.lucene.store.RAMOutputStream@3afabf87); >>> now aborting this merge to prevent index corruption >>> [junit4] > at >>> __randomizedtesting.SeedInfo.seed([70486912DEA23EAA:FF309021F56884D9]:0) >>> [junit4] > at >>> org.apache.lucene.codecs.lucene40.Lucene40StoredFieldsWriter.finish(Lucene40StoredFieldsWriter.java:233) >>> [junit4] > at >>> org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:96) >>> [junit4] > at >>> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:415) >>> [junit4] > at >>> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:512) >>> [junit4] > at >>> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622) >>> [junit4] > at >>> org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3203) >>> [junit4] > at >>> org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3179) >>> [junit4] > at >>> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:995) >>> [junit4] > at >>> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:939) >>> [junit4] > at >>> org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads(TestIndexWriterWithThreads.java:168) >>> [junit4] > at java.lang.Thread.run(Thread.java:724) >>> [junit4] 2> NOTE: test params are: codec=Lucene40, >>> sim=DefaultSimilarity, locale=ko, timezone=Etc/GMT+9 >>> [junit4] 2> NOTE: Linux 3.2.0-26-generic amd64/Oracle Corporation >>> 1.7.0_25 (64-bit)/cpus=8,threads=1,free=241749984,total=258539520 >>> [junit4] 2> NOTE: All tests run in this JVM: [TestSloppyMath, >>> TestRamUsageEstimatorOnWildAnimals, TestReusableStringReader, >>> TestByteArrayDataInput, TestSentinelIntSet, TestIndexWriterWithThreads] >>> [junit4] Completed on J2 in 14.03s, 12 tests, 1 error <<< FAILURES! >>> >>> [...truncated 1260 lines...] >>> BUILD FAILED >>> /var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/build.xml:49: >>> The following error occurred while executing this line: >>> /var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/common-build.xml:1297: >>> The following error occurred while executing this line: >>> /var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/common-build.xml:921: >>> There were test failures: 400 suites, 3109 tests, 1 error, 97 ignored (87 >>> assumptions) >>> >>> Total time: 5 minutes 42 seconds >>> Build step 'Invoke Ant' marked build as failure >>> Recording test results >>> Email was triggered for: Failure >>> Sending email for trigger: Failure >>> >>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
