[ https://issues.apache.org/jira/browse/LUCENE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14392855#comment-14392855 ]
Shai Erera commented on LUCENE-6386: ------------------------------------ I thought we only write the final segments as .cfs if needed? Does what you say mean the following happens: {noformat} source: [s1,s2] [s3,s4] [s5,s6] [s7,s8] level1: [s9,s10], [s11,s12] level1: [s9.cfs,s10.cfs], [s11.cfs,s12.cfs] level2: [s13,s14] level2: [s13.cfs,s14.cfs] level3: [s15] final: [s15.cfs] {noformat} So when it gets to create level3, with the {{level2.cfs}} written, the index contains {{source}} (1X), {{level2.cfs}} (1X), {{level3}} (1X) and {{level3.cfs}} (1X) -- that's 4X indeed. Is there a reason why we don't delete {{level2.cfs}} after creating {{level3}}? If not, can we fix that and reduce 1X here? > TestIndexWriterForceMerge still unreliable in NIGHTLY > ----------------------------------------------------- > > Key: LUCENE-6386 > URL: https://issues.apache.org/jira/browse/LUCENE-6386 > Project: Lucene - Core > Issue Type: Bug > Reporter: Robert Muir > Assignee: Michael McCandless > Fix For: Trunk, 5.1 > > Attachments: LUCENE-6386.patch > > > Discovered by ryan beasting (trunk): > ant test -Dtestcase=TestIndexWriterForceMerge > -Dtests.method=testForceMergeTempSpaceUsage -Dtests.seed=DC9ADB74850A581B > -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true > -Dtests.locale=sr__#Latn -Dtests.timezone=Indian/Chagos -Dtests.asserts=true > -Dtests.file.encoding=US-ASCII > {noformat} > [junit4] 2> NOTE: reproduce with: ant test > -Dtestcase=TestIndexWriterForceMerge > -Dtests.method=testForceMergeTempSpaceUsage -Dtests.seed=DC9ADB74850A581B > -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true > -Dtests.locale=sr__#Latn -Dtests.timezone=Indian/Chagos -Dtests.asserts=true > -Dtests.file.encoding=US-ASCII > [junit4] FAILURE 1.20s | > TestIndexWriterForceMerge.testForceMergeTempSpaceUsage <<< > [junit4] > Throwable #1: java.lang.AssertionError: forceMerge used too > much temporary space: starting usage was 291570 bytes; final usage was 262469 > bytes; max temp usage was 1079501 but should have been 874710 (= 3X starting > usage), BEFORE= > [junit4] > _u.scf 146329 > [junit4] > _u.si 635 > [junit4] > |- (inside compound file) _u.fld 2214 > [junit4] > |- (inside compound file) _u.inf 392 > [junit4] > |- (inside compound file) _u.len 2381 > [junit4] > |- (inside compound file) _u.pst 36758 > [junit4] > |- (inside compound file) _u.vec 104144 > [junit4] > _s.pst 1338 > [junit4] > _s.inf 392 > [junit4] > _s.fld 94 > [junit4] > _s.len 221 > [junit4] > _s.vec 3744 > [junit4] > _s.si 624 > [junit4] > _t.fld 94 > [junit4] > _t.len 221 > [junit4] > _t.pst 1338 > [junit4] > _t.inf 392 > [junit4] > _t.vec 3744 > [junit4] > _t.si 624 > [junit4] > _v.fld 94 > [junit4] > _v.pst 1338 > [junit4] > _v.inf 392 > [junit4] > _v.vec 3744 > [junit4] > _v.si 624 > [junit4] > _v.len 221 > [junit4] > _w.len 221 > [junit4] > _w.pst 1338 > [junit4] > _w.inf 392 > [junit4] > _w.fld 94 > [junit4] > _w.si 624 > [junit4] > _w.vec 3744 > [junit4] > _x.vec 3744 > [junit4] > _x.inf 392 > [junit4] > _x.pst 1338 > [junit4] > _x.fld 94 > [junit4] > _x.si 624 > [junit4] > _x.len 221 > [junit4] > _y.fld 94 > [junit4] > _y.pst 1338 > [junit4] > _y.inf 392 > [junit4] > _y.si 624 > [junit4] > _y.vec 3744 > [junit4] > _y.len 221 > [junit4] > _z.fld 94 > [junit4] > _z.pst 1338 > [junit4] > _z.inf 392 > [junit4] > _z.len 221 > [junit4] > _z.vec 3744 > [junit4] > _z.si 624 > [junit4] > _10.si 630 > [junit4] > _10.fld 94 > [junit4] > _10.pst 1338 > [junit4] > _10.inf 392 > [junit4] > _10.vec 3744 > [junit4] > _10.len 221 > [junit4] > _11.len 221 > [junit4] > _11.si 630 > [junit4] > _11.vec 3744 > [junit4] > _11.pst 1338 > [junit4] > _11.inf 392 > [junit4] > _11.fld 94 > [junit4] > _12.vec 3744 > [junit4] > _12.si 630 > [junit4] > _12.len 221 > [junit4] > _12.fld 94 > [junit4] > _12.pst 1338 > [junit4] > _12.inf 392 > [junit4] > _13.fld 94 > [junit4] > _13.vec 3744 > [junit4] > _13.si 630 > [junit4] > _13.pst 1338 > [junit4] > _13.inf 392 > [junit4] > _13.len 221 > [junit4] > _14.fld 94 > [junit4] > _14.pst 1338 > [junit4] > _14.inf 392 > [junit4] > _14.si 630 > [junit4] > _14.vec 3744 > [junit4] > _14.len 221 > [junit4] > _15.len 221 > [junit4] > _15.vec 3744 > [junit4] > _15.si 630 > [junit4] > _15.pst 1338 > [junit4] > _15.inf 392 > [junit4] > _15.fld 94 > [junit4] > _16.vec 3744 > [junit4] > _16.len 221 > [junit4] > _16.fld 94 > [junit4] > _16.si 630 > [junit4] > _16.pst 1338 > [junit4] > _16.inf 392 > [junit4] > _17.vec 3744 > [junit4] > _17.pst 1338 > [junit4] > _17.inf 392 > [junit4] > _17.len 221 > [junit4] > _17.si 630 > [junit4] > _17.fld 94 > [junit4] > _18.pst 1338 > [junit4] > _18.inf 392 > [junit4] > _18.len 221 > [junit4] > _18.vec 3744 > [junit4] > _18.si 630 > [junit4] > _18.fld 94 > [junit4] > _19.fld 94 > [junit4] > _19.si 630 > [junit4] > _19.len 221 > [junit4] > _19.vec 3744 > [junit4] > _19.pst 1338 > [junit4] > _19.inf 392 > [junit4] > _1a.fld 94 > [junit4] > _1a.pst 1338 > [junit4] > _1a.inf 392 > [junit4] > _1a.len 221 > [junit4] > _1a.vec 3744 > [junit4] > _1a.si 630 > [junit4] > _1b.fld 94 > [junit4] > _1b.si 630 > [junit4] > _1b.pst 1338 > [junit4] > _1b.inf 392 > [junit4] > _1b.vec 3744 > [junit4] > _1b.len 221 > [junit4] > _1c.vec 3744 > [junit4] > _1c.pst 1338 > [junit4] > _1c.inf 392 > [junit4] > _1c.fld 94 > [junit4] > _1c.len 221 > [junit4] > _1c.si 630 > [junit4] > _1d.pst 1338 > [junit4] > _1d.inf 392 > [junit4] > _1d.fld 94 > [junit4] > _1d.len 221 > [junit4] > _1d.vec 3744 > [junit4] > _1d.si 630 > [junit4] > _1e.fld 94 > [junit4] > _1e.pst 1338 > [junit4] > _1e.inf 392 > [junit4] > _1e.len 221 > [junit4] > _1e.vec 3744 > [junit4] > _1e.si 630 > [junit4] > _1f.fld 40 > [junit4] > _1f.pst 195 > [junit4] > _1f.inf 392 > [junit4] > _1f.len 149 > [junit4] > _1f.vec 405 > [junit4] > _1f.si 629 > [junit4] > AFTER= > [junit4] > _1j.si 635 > [junit4] > _1j.scf 261701 > [junit4] > |- (inside compound file) _1j.fld 3982 > [junit4] > |- (inside compound file) _1j.inf 392 > [junit4] > |- (inside compound file) _1j.len 4149 > [junit4] > |- (inside compound file) _1j.pst 66151 > [junit4] > |- (inside compound file) _1j.vec 186577 > [junit4] > at > __randomizedtesting.SeedInfo.seed([DC9ADB74850A581B:C6581887EB1ABBDC]:0) > [junit4] > at > org.apache.lucene.index.TestIndexWriterForceMerge.testForceMergeTempSpaceUsage(TestIndexWriterForceMerge.java:199) > [junit4] > at java.lang.Thread.run(Thread.java:745) > [junit4] 2> NOTE: test params are: codec=SimpleText, > sim=RandomSimilarityProvider(queryNorm=false,coord=no): {id=IB SPL-D2, > content=DFR I(F)B1}, locale=sr__#Latn, timezone=Indian/Chagos > [junit4] 2> NOTE: Linux 3.13.0-45-generic amd64/Oracle Corporation > 1.8.0_40-ea (64-bit)/cpus=8,threads=1,free=208092000,total=253231104 > [junit4] 2> NOTE: All tests run in this JVM: [TestIndexWriterForceMerge] > [junit4] Completed in 1.54s, 1 test, 1 failure <<< FAILURES! > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org