I was unable to reproduce this OOM. But: it looks like it was happening due to RamUsageEstimator's overhead of tracking which objects it has seen ... so I committed a fix to do its own rough accounting.
Mike McCandless http://blog.mikemccandless.com On Tue, Sep 4, 2012 at 5:42 PM, Policeman Jenkins Server <[email protected]> wrote: > Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux/863/ > Java: 64bit/ibm-j9-jdk6 > -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;} > > 1 tests failed. > REGRESSION: org.apache.lucene.index.TestPostingsFormat.test > > Error Message: > > > Stack Trace: > java.lang.OutOfMemoryError > at > __randomizedtesting.SeedInfo.seed([2945983AC7B3782C:A111A7E0694F15D4]:0) > at > org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.allocateBuffers(RamUsageEstimator.java:745) > at > org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.expandAndRehash(RamUsageEstimator.java:719) > at > org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.add(RamUsageEstimator.java:661) > at > org.apache.lucene.util.RamUsageEstimator.measureObjectSize(RamUsageEstimator.java:420) > at > org.apache.lucene.util.RamUsageEstimator.sizeOf(RamUsageEstimator.java:333) > at > org.apache.lucene.index.TestPostingsFormat.buildIndex(TestPostingsFormat.java:386) > at > org.apache.lucene.index.TestPostingsFormat.test(TestPostingsFormat.java:926) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:600) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:345) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:769) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:429) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:40) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) > at > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:345) > at java.lang.Thread.run(Thread.java:736) > > > > > Build Log: > [...truncated 275 lines...] > [junit4:junit4] Suite: org.apache.lucene.index.TestPostingsFormat > [junit4:junit4] 2> NOTE: reproduce with: ant test > -Dtestcase=TestPostingsFormat -Dtests.method=test > -Dtests.seed=2945983AC7B3782C -Dtests.multiplier=3 -Dtests.slow=true > -Dtests.locale=en_GB_EURO -Dtests.timezone=Asia/Manila > -Dtests.file.encoding=ISO-8859-1 > [junit4:junit4] ERROR 14.7s J1 | TestPostingsFormat.test <<< > [junit4:junit4] > Throwable #1: java.lang.OutOfMemoryError > [junit4:junit4] > at > __randomizedtesting.SeedInfo.seed([2945983AC7B3782C:A111A7E0694F15D4]:0) > [junit4:junit4] > at > org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.allocateBuffers(RamUsageEstimator.java:745) > [junit4:junit4] > at > org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.expandAndRehash(RamUsageEstimator.java:719) > [junit4:junit4] > at > org.apache.lucene.util.RamUsageEstimator$IdentityHashSet.add(RamUsageEstimator.java:661) > [junit4:junit4] > at > org.apache.lucene.util.RamUsageEstimator.measureObjectSize(RamUsageEstimator.java:420) > [junit4:junit4] > at > org.apache.lucene.util.RamUsageEstimator.sizeOf(RamUsageEstimator.java:333) > [junit4:junit4] > at > org.apache.lucene.index.TestPostingsFormat.buildIndex(TestPostingsFormat.java:386) > [junit4:junit4] > at > org.apache.lucene.index.TestPostingsFormat.test(TestPostingsFormat.java:926) > [junit4:junit4] > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > [junit4:junit4] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48) > [junit4:junit4] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > [junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:600) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) > [junit4:junit4] > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:345) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:769) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:429) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) > [junit4:junit4] > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:40) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:345) > [junit4:junit4] > at java.lang.Thread.run(Thread.java:736) > [junit4:junit4] 2> NOTE: test params are: codec=Lucene40: > {yucyqgi=PostingsFormat(name=TestBloomFilteredLucene40Postings), > gqwopmb=PostingsFormat(name=TestBloomFilteredLucene40Postings), > fc=Pulsing40(freqCutoff=7 minBlockSize=18 maxBlockSize=55), > apzcf=MockFixedIntBlock(blockSize=739), > vitao=PostingsFormat(name=TestBloomFilteredLucene40Postings)}, > sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, > locale=en_GB_EURO, timezone=Asia/Manila > [junit4:junit4] 2> NOTE: Linux 3.2.0-29-generic amd64/IBM Corporation 1.6.0 > (64-bit)/cpus=8,threads=1,free=51955544,total=536870912 > [junit4:junit4] 2> NOTE: All tests run in this JVM: [TestAttributeSource, > TestNGramPhraseQuery, TestLockFactory, TestBooleanMinShouldMatch, > TestDateTools, TestIOUtils, TestPositionIncrement, TestIndexInput, > TestFuzzyQuery, TestCollectionUtil, TestBasicOperations, > TestForceMergeForever, Nested, Nested, TestPrefixCodedTerms, > TestNRTReaderWithThreads, Nested1, TestConjunctions, TestPostingsFormat] > [junit4:junit4] Completed on J1 in 21.42s, 1 test, 1 error <<< FAILURES! > > [...truncated 903 lines...] > BUILD FAILED > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:29: The following > error occurred while executing this line: > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build.xml:49: The > following error occurred while executing this line: > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1056: > The following error occurred while executing this line: > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:741: > There were test failures: 316 suites, 1793 tests, 1 error, 17 ignored (9 > assumptions) > > Total time: 7 minutes 15 seconds > Build step 'Invoke Ant' marked build as failure > Recording test results > Description set: Java: 64bit/ibm-j9-jdk6 > -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;} > Email was triggered for: Failure > Sending email for trigger: Failure > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
