I think we have to somehow bound the parameters to NgramTokenizer in this test. when you have these huge min/max its like 16, 95, its like making 80 separate indexes in whats happening.
THen followup with a shinglefilter and its tooo much. Its happened twice today. On Tue, Mar 25, 2014 at 8:24 PM, <[email protected]> wrote: > Build: builds.flonkings.com/job/Lucene-trunk-Linux-java7-64-analyzers/699/ > > 1 tests failed. > REGRESSION: > org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings > > Error Message: > Java heap space > > Stack Trace: > java.lang.OutOfMemoryError: Java heap space > at > __randomizedtesting.SeedInfo.seed([A9AFA8E25020FEEE:C3F417F3096EDE1D]:0) > at java.util.Arrays.copyOfRange(Arrays.java:2694) > at java.lang.String.<init>(String.java:203) > at > org.apache.lucene.analysis.tokenattributes.CharTermAttributeImpl.toString(CharTermAttributeImpl.java:267) > at > org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:702) > at > org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:611) > at > org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:510) > at > org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:920) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:826) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:862) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876) > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) > > > > > Build Log: > [...truncated 763 lines...] > [junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains > [junit4] 2> TEST FAIL: useCharFilter=false text='hpmy > \uf92d\u0012\u06c7\u1a6e!\u0334\uda50\udf32g ljndl \ufe5b\ufe50 㮤 k lu > zpl ce \ue537\ud8d0\udea8 \udae1\ude13 sz ?\u0233)\u0144 cphxoa > p\ud8f3\ude3c \ucce6 uuuvo uyjrdgmnvcww \u1386\u1398\u1381' > [junit4] 2> Exception from random analyzer: > [junit4] 2> charfilters= > [junit4] 2> tokenizer= > [junit4] 2> org.apache.lucene.analysis.ngram.NGramTokenizer(LUCENE_50, > org.apache.lucene.util.AttributeSource$AttributeFactory$DefaultAttributeFactory@7b7566ad, > 16, 95) > [junit4] 2> filters= > [junit4] 2> > org.apache.lucene.analysis.br.BrazilianStemFilter(ValidatingTokenFilter@7ddb500c > > term=,bytes=[],positionIncrement=1,positionLength=1,startOffset=0,endOffset=0,keyword=false) > [junit4] 2> > org.apache.lucene.analysis.MockTokenFilter(ValidatingTokenFilter@662f21eb > term=,bytes=[],positionIncrement=1,positionLength=1,startOffset=0,endOffset=0,keyword=false, > initial state: 1 > [junit4] 2> state 0 [accept]: > [junit4] 2> \\U00000000-\\U00000008 -> 0 > [junit4] 2> \\U0000000b-\\U0000000c -> 0 > [junit4] 2> \\U0000000e-\\U0000001f -> 0 > [junit4] 2> !-\\U0010ffff -> 0 > [junit4] 2> state 1 [reject]: > [junit4] 2> \\U00000000-\\U00000008 -> 0 > [junit4] 2> \\U0000000b-\\U0000000c -> 0 > [junit4] 2> \\U0000000e-\\U0000001f -> 0 > [junit4] 2> !-\\U0010ffff -> 0 > [junit4] 2> ) > [junit4] 2> > org.apache.lucene.analysis.shingle.ShingleFilter(ValidatingTokenFilter@77f11652 > > term=,bytes=[],positionIncrement=1,positionLength=1,startOffset=0,endOffset=0,keyword=false,type=word, > 92) > [junit4] 2> > org.apache.lucene.analysis.sinks.TeeSinkTokenFilter(ValidatingTokenFilter@3599f0e5 > > term=,bytes=[],positionIncrement=1,positionLength=1,startOffset=0,endOffset=0,keyword=false,type=word) > [junit4] 2> offsetsAreCorrect=true > [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestRandomChains > -Dtests.method=testRandomChainsWithLargeStrings -Dtests.seed=A9AFA8E25020FEEE > -Dtests.slow=true -Dtests.locale=is_IS -Dtests.timezone=Europe/Athens > -Dtests.file.encoding=US-ASCII > [junit4] ERROR 20.2s J2 | > TestRandomChains.testRandomChainsWithLargeStrings <<< > [junit4] > Throwable #1: java.lang.OutOfMemoryError: Java heap space > [junit4] > at > __randomizedtesting.SeedInfo.seed([A9AFA8E25020FEEE:C3F417F3096EDE1D]:0) > [junit4] > at java.util.Arrays.copyOfRange(Arrays.java:2694) > [junit4] > at java.lang.String.<init>(String.java:203) > [junit4] > at > org.apache.lucene.analysis.tokenattributes.CharTermAttributeImpl.toString(CharTermAttributeImpl.java:267) > [junit4] > at > org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:702) > [junit4] > at > org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:611) > [junit4] > at > org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:510) > [junit4] > at > org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:920) > [junit4] 2> NOTE: test params are: codec=Lucene42, > sim=DefaultSimilarity, locale=is_IS, timezone=Europe/Athens > [junit4] 2> NOTE: Linux 3.2.0-26-generic amd64/Oracle Corporation > 1.7.0_25 (64-bit)/cpus=8,threads=1,free=174932808,total=262930432 > [junit4] 2> NOTE: All tests run in this JVM: [TestHunspellStemFilter, > TestCJKBigramFilter, TestSynonymFilterFactory, TestCharArraySet, > TestTurkishAnalyzer, TestCJKWidthFilter, TestCzechAnalyzer, TestNGramFilters, > TestPathHierarchyTokenizer, TestCondition, TestApostropheFilter, > TestRandomChains] > [junit4] Completed on J2 in 51.36s, 2 tests, 1 error <<< FAILURES! > > [...truncated 292 lines...] > BUILD FAILED > /var/lib/jenkins/workspace/Lucene-trunk-Linux-java7-64-analyzers/checkout/lucene/module-build.xml:60: > The following error occurred while executing this line: > /var/lib/jenkins/workspace/Lucene-trunk-Linux-java7-64-analyzers/checkout/lucene/common-build.xml:1275: > The following error occurred while executing this line: > /var/lib/jenkins/workspace/Lucene-trunk-Linux-java7-64-analyzers/checkout/lucene/common-build.xml:907: > There were test failures: 249 suites, 1323 tests, 1 error, 1 ignored > > Total time: 2 minutes 32 seconds > Build step 'Invoke Ant' marked build as failure > Recording test results > Email was triggered for: Failure > Sending email for trigger: Failure > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
