This patch also works without the line: + mp.setMaxCFSSegmentSizeMB(Double.POSITIVE_INFINITY);
The reason why it fixes is changed order of newLogMergePolicy() and newIndexWriterConfig() -> causing different settings! :-) So it's just a side-effect of changing the order of the 2 calls. The reason why this bug appears is explained in the issue, you reopened: I added more randomization to LTC, causeing the cfsRatio (randomized already for TieredMP) to be randomized, too. Without you patch, the ratio was 0.82, causing CFS files to be created. CFS file creation causes additional disk space used. The correct fix is: newLogMergePolicy(false) -> no CFS allowed. ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: [email protected] > -----Original Message----- > From: Robert Muir [mailto:[email protected]] > Sent: Friday, August 24, 2012 2:42 PM > To: [email protected] > Subject: Re: Build failed in Jenkins: Lucene-trunk-Linux-Java7-64-test-only > #3206 > > I think this is a bug in LUCENE-4323. > > I'm suspicious this option is working as documented: these directories are > only > a few KB but its kicking in even though the limit is like 8MB. > > So the following patch prevents the issue (other disk full tests might need > it to), > but I dont want to commit this until someone can confirm > setMaxCFSSegmentSizeMB is actually working in MB! > > Index: src/test/org/apache/lucene/index/TestIndexWriterOnDiskFull.java > ================================================================ > === > --- src/test/org/apache/lucene/index/TestIndexWriterOnDiskFull.java > (revision > 1376894) > +++ src/test/org/apache/lucene/index/TestIndexWriterOnDiskFull.java > (working > copy) > @@ -249,7 +249,9 @@ > > // Make a new dir that will enforce disk usage: > MockDirectoryWrapper dir = new MockDirectoryWrapper(random(), new > RAMDirectory(startDir, newIOContext(random()))); > - writer = new IndexWriter(dir, > newIndexWriterConfig(TEST_VERSION_CURRENT, new > MockAnalyzer(random())).setOpenMode(OpenMode.APPEND).setMergePolicy( > newLogMergePolicy())); > + LogMergePolicy mp = newLogMergePolicy(); > + mp.setMaxCFSSegmentSizeMB(Double.POSITIVE_INFINITY); > + writer = new IndexWriter(dir, > newIndexWriterConfig(TEST_VERSION_CURRENT, new > MockAnalyzer(random())).setOpenMode(OpenMode.APPEND).setMergePolicy( > mp)); > IOException err = null; > > MergeScheduler ms = writer.getConfig().getMergeScheduler(); > > > On Fri, Aug 24, 2012 at 2:55 AM, Dawid Weiss <[email protected]> > wrote: > > Don't know what this is -- should we file an issue? > > > > [junit4:junit4] Suite: org.apache.lucene.index.TestIndexWriterOnDiskFull > > [junit4:junit4] 2> NOTE: reproduce with: ant test > > -Dtestcase=TestIndexWriterOnDiskFull > > -Dtests.method=testAddIndexOnDiskFull -Dtests.seed=17160CAE40DA6C29 > > -Dtests.slow=true -Dtests.locale=hr_HR -Dtests.timezone=Etc/GMT+12 > > -Dtests.file.encoding=UTF-8 > > [junit4:junit4] FAILURE 1.67s J6 | > > TestIndexWriterOnDiskFull.testAddIndexOnDiskFull <<< > > [junit4:junit4] > Throwable #1: java.lang.AssertionError: max free > > Directory space required exceeded 1X the total input index sizes > > during addIndexes(Directory[]) + forceMerge(1): max temp usage = 31193 > > bytes vs limit=30618; starting disk usage = 4882 bytes; input index > > disk usage = 10427 bytes > > [junit4:junit4] > at > > > __randomizedtesting.SeedInfo.seed([17160CAE40DA6C29:6BC9BD910D6BC49E > ]:0) > > [junit4:junit4] > at org.junit.Assert.fail(Assert.java:93) > > [junit4:junit4] > at org.junit.Assert.assertTrue(Assert.java:43) > > [junit4:junit4] > at > > > org.apache.lucene.index.TestIndexWriterOnDiskFull.testAddIndexOnDiskFull(Te > stIndexWriterOnDiskFull.java:424) > > [junit4:junit4] > at > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > [junit4:junit4] > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: > 57) > > [junit4:junit4] > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI > mpl.java:43) > > [junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:601) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRu > nner.java:1559) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(Randomiz > edRunner.java:79) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Randomiz > edRunner.java:737) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Randomiz > edRunner.java:773) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Randomiz > edRunner.java:787) > > [junit4:junit4] > at > > > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSet > upTeardownChained.java:50) > > [junit4:junit4] > at > > > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCach > eSanity.java:51) > > [junit4:junit4] > at > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte > rRule.java:45) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev > aluate(SystemPropertiesInvariantRule.java:55) > > [junit4:junit4] > at > > > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThrea > dAndTestName.java:48) > > [junit4:junit4] > at > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgn > oreAfterMaxFailures.java:70) > > [junit4:junit4] > at > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.ja > va:48) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stateme > ntAdapter.java:36) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run > (ThreadLeakControl.java:345) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(T > hreadLeakControl.java:769) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLea > kControl.java:429) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Rando > mizedRunner.java:746) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(Randomiz > edRunner.java:648) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Randomiz > edRunner.java:682) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Randomiz > edRunner.java:693) > > [junit4:junit4] > at > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte > rRule.java:45) > > [junit4:junit4] > at > > > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClass > Name.java:42) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev > aluate(SystemPropertiesInvariantRule.java:55) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethod > sRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethod > sRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > [junit4:junit4] > at > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertio > nsRequired.java:40) > > [junit4:junit4] > at > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.ja > va:48) > > [junit4:junit4] > at > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgn > oreAfterMaxFailures.java:70) > > [junit4:junit4] > at > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTest > Suites.java:55) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stateme > ntAdapter.java:36) > > [junit4:junit4] > at > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run > (ThreadLeakControl.java:345) > > [junit4:junit4] > at java.lang.Thread.run(Thread.java:722) > > [junit4:junit4] 2> NOTE: test params are: codec=Asserting, > > sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {id=DFR > > I(ne)1, f=IB SPL-LZ(0.3), content=IB SPL-L2, field=DFR I(ne)2}, > > locale=hr_HR, timezone=Etc/GMT+12 > > [junit4:junit4] 2> NOTE: Linux 3.2.0-26-generic amd64/Oracle > > Corporation 1.7.0_05 > > (64-bit)/cpus=8,threads=1,free=159881760,total=262406144 > > [junit4:junit4] 2> NOTE: All tests run in this JVM: > > [TestLucene40PostingsReader, Nested3, Nested1, Nested2, > > TestStressAdvance, TestCollectionUtil, TestIndexWriterOnDiskFull] > > [junit4:junit4] Completed on J6 in 4.36s, 4 tests, 1 failure <<< FAILURES! > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] For > > additional commands, e-mail: [email protected] > > > > > > -- > lucidworks.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] For additional > commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
