shubhamsrkdev commented on issue #15662: URL: https://github.com/apache/lucene/issues/15662#issuecomment-3841461819
I tried git bisecting on the commits: https://github.com/apache/lucene/compare/2f9aa8ae26d6c1087884c734e1b3d137bd8c6601...6a6b753e3080725921a07b0a214963d8ff639eea and running luceneutil benchmarks on `wikimediumall` dataset. It is not possible to do a direct git bisect because of [this CFS change](https://github.com/apache/lucene/pull/15295). Since lucene util also [changed](https://github.com/mikemccand/luceneutil/pull/510), so newer (present) lucene util is not compatible with pre CFS change and vice-versa. Therefore, I tried bisecting between the [CFS code merge](https://github.com/apache/lucene/pull/15295) and current mainline where I see the original improvements: `python src/python/localrun.py -source wikimediumall ` ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value range 2167.24 (5.9%) 2054.83 (5.5%) -5.2% ( -15% - 6%) 0.004 BrowseDayOfYearSSDVFacets 4.68 (14.2%) 4.49 (6.3%) -4.2% ( -21% - 19%) 0.230 BrowseDateSSDVFacets 0.91 (7.9%) 0.88 (8.9%) -3.2% ( -18% - 14%) 0.235 BrowseDateTaxoFacets 3.28 (11.6%) 3.19 (4.9%) -3.0% ( -17% - 15%) 0.287 BrowseDayOfYearTaxoFacets 3.30 (10.9%) 3.21 (4.8%) -2.8% ( -16% - 14%) 0.288 BrowseRandomLabelTaxoFacets 2.45 (5.6%) 2.41 (2.0%) -1.8% ( -8% - 6%) 0.175 OrHighMedDayTaxoFacets 5.22 (2.2%) 5.17 (3.0%) -0.9% ( -5% - 4%) 0.270 TermDTSort 306.21 (2.3%) 303.66 (2.9%) -0.8% ( -5% - 4%) 0.308 BrowseRandomLabelSSDVFacets 3.13 (5.1%) 3.11 (5.4%) -0.7% ( -10% - 10%) 0.668 HighTermDayOfYearSort 320.25 (1.9%) 319.08 (2.6%) -0.4% ( -4% - 4%) 0.613 HighIntervalsOrdered 44.29 (6.8%) 44.14 (6.7%) -0.3% ( -12% - 14%) 0.871 Fuzzy2 70.51 (1.2%) 70.29 (1.5%) -0.3% ( -2% - 2%) 0.475 IntSet 781.91 (6.1%) 780.28 (7.0%) -0.2% ( -12% - 13%) 0.920 MedTermDayTaxoFacets 22.46 (1.5%) 22.43 (1.6%) -0.2% ( -3% - 3%) 0.731 HighSpanNear 65.50 (1.9%) 65.43 (3.8%) -0.1% ( -5% - 5%) 0.905 Fuzzy1 94.09 (1.6%) 94.00 (2.0%) -0.1% ( -3% - 3%) 0.869 MedSpanNear 11.72 (2.6%) 11.72 (3.6%) -0.0% ( -6% - 6%) 0.978 AndHighHighDayTaxoFacets 12.10 (1.8%) 12.11 (2.1%) 0.1% ( -3% - 4%) 0.877 MedIntervalsOrdered 13.35 (4.9%) 13.37 (4.3%) 0.1% ( -8% - 9%) 0.933 Respell 66.36 (2.1%) 66.44 (1.5%) 0.1% ( -3% - 3%) 0.822 LowIntervalsOrdered 70.41 (2.8%) 70.57 (2.3%) 0.2% ( -4% - 5%) 0.782 HighTermTitleBDVSort 41.86 (1.6%) 42.05 (1.4%) 0.5% ( -2% - 3%) 0.319 IntNRQ 378.38 (2.1%) 380.53 (2.0%) 0.6% ( -3% - 4%) 0.384 MedPhrase 45.18 (1.3%) 45.45 (1.2%) 0.6% ( -1% - 3%) 0.124 LowSpanNear 113.96 (1.6%) 114.69 (2.6%) 0.6% ( -3% - 4%) 0.347 BrowseMonthTaxoFacets 2.77 (0.4%) 2.79 (2.0%) 0.7% ( -1% - 3%) 0.161 HighSloppyPhrase 26.14 (3.2%) 26.42 (3.7%) 1.1% ( -5% - 8%) 0.317 AndHighMedDayTaxoFacets 149.76 (0.9%) 151.52 (0.9%) 1.2% ( 0% - 3%) 0.000 MedSloppyPhrase 15.21 (5.2%) 15.44 (5.1%) 1.5% ( -8% - 12%) 0.353 PKLookup 200.48 (3.8%) 203.61 (3.1%) 1.6% ( -5% - 8%) 0.150 HighPhrase 156.56 (1.4%) 159.10 (1.3%) 1.6% ( -1% - 4%) 0.000 LowSloppyPhrase 160.25 (1.8%) 163.71 (2.2%) 2.2% ( -1% - 6%) 0.001 Wildcard 108.28 (9.6%) 110.64 (8.3%) 2.2% ( -14% - 22%) 0.441 AndHighHigh 330.04 (1.4%) 337.48 (1.7%) 2.3% ( 0% - 5%) 0.000 BrowseMonthSSDVFacets 4.57 (11.9%) 4.69 (14.8%) 2.6% ( -21% - 33%) 0.535 OrHighNotHigh 468.51 (5.0%) 483.74 (4.1%) 3.3% ( -5% - 13%) 0.025 OrHighHigh 338.29 (3.8%) 349.31 (2.5%) 3.3% ( -2% - 9%) 0.001 HighTerm 851.68 (4.3%) 880.79 (5.4%) 3.4% ( -6% - 13%) 0.027 MedTerm 1046.18 (3.3%) 1088.33 (3.7%) 4.0% ( -2% - 11%) 0.000 OrNotHighMed 577.71 (3.3%) 602.14 (3.1%) 4.2% ( -2% - 10%) 0.000 HighTermMonthSort 1370.77 (3.3%) 1429.12 (3.0%) 4.3% ( -1% - 10%) 0.000 OrHighNotLow 1043.58 (4.3%) 1090.04 (4.6%) 4.5% ( -4% - 13%) 0.002 OrNotHighHigh 650.22 (5.0%) 682.80 (5.7%) 5.0% ( -5% - 16%) 0.003 AndHighMed 569.05 (1.4%) 598.19 (1.8%) 5.1% ( 1% - 8%) 0.000 OrHighNotMed 781.18 (3.6%) 824.80 (3.7%) 5.6% ( -1% - 13%) 0.000 LowPhrase 526.37 (2.5%) 557.54 (2.7%) 5.9% ( 0% - 11%) 0.000 OrHighMed 677.35 (2.6%) 720.99 (2.1%) 6.4% ( 1% - 11%) 0.000 LowTerm 1618.76 (2.9%) 1762.64 (2.3%) 8.9% ( 3% - 14%) 0.000 OrNotHighLow 1358.39 (2.8%) 1485.57 (2.3%) 9.4% ( 4% - 14%) 0.000 OrHighLow 998.42 (3.2%) 1098.47 (3.0%) 10.0% ( 3% - 16%) 0.000 Prefix3 1021.33 (2.4%) 1130.88 (3.3%) 10.7% ( 4% - 16%) 0.000 HighTermTitleSort 167.99 (1.8%) 186.12 (2.2%) 10.8% ( 6% - 15%) 0.000 AndHighLow 1290.62 (3.1%) 1438.45 (4.0%) 11.5% ( 4% - 19%) 0.000 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
