shubhamsrkdev commented on issue #15662: URL: https://github.com/apache/lucene/issues/15662#issuecomment-3841482789
Owing to this: >It is not possible to do a direct git bisect because of https://github.com/apache/lucene/pull/15295. Since lucene util also https://github.com/mikemccand/luceneutil/pull/510, so newer (present) lucene util is not compatible with pre CFS change and vice-versa. To test the [CFS code merge](https://github.com/apache/lucene/pull/15295), I removed the [CFS method calls from lucene utils](https://github.com/mikemccand/luceneutil/pull/510), and then ran benchmarks between the [pre 22/01 commit](https://github.com/apache/lucene/commit/2f9aa8ae26d6c1087884c734e1b3d137bd8c6601) and [CFS one](https://github.com/apache/lucene/commit/6643a5dbe1ed6d3a4b933c90beed60cfe44b5992) It also shows no regressions. (I am not sure if this is the correct/best method to do this): ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value MedSloppyPhrase 45.23 (3.5%) 43.42 (8.1%) -4.0% ( -15% - 7%) 0.043 OrNotHighHigh 389.30 (4.4%) 377.75 (4.8%) -3.0% ( -11% - 6%) 0.041 OrHighNotHigh 603.76 (3.7%) 587.83 (3.4%) -2.6% ( -9% - 4%) 0.019 HighTerm 875.27 (4.2%) 854.98 (7.0%) -2.3% ( -13% - 9%) 0.207 OrNotHighLow 1404.62 (2.7%) 1375.72 (2.8%) -2.1% ( -7% - 3%) 0.018 Wildcard 228.83 (5.6%) 224.44 (5.8%) -1.9% ( -12% - 10%) 0.287 IntSet 807.93 (4.5%) 793.28 (5.6%) -1.8% ( -11% - 8%) 0.256 LowSloppyPhrase 116.93 (3.1%) 114.83 (5.0%) -1.8% ( -9% - 6%) 0.175 OrNotHighMed 727.53 (3.2%) 717.41 (3.1%) -1.4% ( -7% - 5%) 0.163 OrHighNotMed 702.45 (5.8%) 692.74 (3.5%) -1.4% ( -10% - 8%) 0.359 AndHighLow 1338.92 (2.3%) 1323.26 (2.4%) -1.2% ( -5% - 3%) 0.118 MedTerm 1137.56 (4.3%) 1125.17 (4.5%) -1.1% ( -9% - 7%) 0.431 HighPhrase 230.70 (1.7%) 228.29 (2.1%) -1.0% ( -4% - 2%) 0.083 MedTermDayTaxoFacets 24.85 (1.3%) 24.64 (1.4%) -0.8% ( -3% - 1%) 0.055 TermDTSort 256.10 (3.8%) 254.16 (3.3%) -0.8% ( -7% - 6%) 0.501 OrHighMedDayTaxoFacets 14.31 (2.4%) 14.21 (1.9%) -0.7% ( -4% - 3%) 0.275 LowIntervalsOrdered 460.54 (5.5%) 457.14 (5.9%) -0.7% ( -11% - 11%) 0.682 OrHighNotLow 980.75 (5.8%) 974.07 (3.8%) -0.7% ( -9% - 9%) 0.661 Prefix3 733.46 (3.0%) 728.66 (2.7%) -0.7% ( -6% - 5%) 0.465 HighIntervalsOrdered 5.96 (4.8%) 5.93 (5.1%) -0.6% ( -10% - 9%) 0.717 BrowseMonthTaxoFacets 2.77 (0.9%) 2.76 (2.7%) -0.5% ( -4% - 3%) 0.460 HighTermTitleSort 148.59 (3.3%) 147.94 (2.7%) -0.4% ( -6% - 5%) 0.647 HighSloppyPhrase 89.69 (2.5%) 89.47 (2.6%) -0.2% ( -5% - 4%) 0.762 LowPhrase 62.31 (2.3%) 62.16 (2.2%) -0.2% ( -4% - 4%) 0.736 MedPhrase 67.23 (1.8%) 67.08 (1.7%) -0.2% ( -3% - 3%) 0.684 MedIntervalsOrdered 61.13 (5.0%) 61.00 (5.6%) -0.2% ( -10% - 10%) 0.896 AndHighHighDayTaxoFacets 37.95 (1.0%) 37.92 (0.9%) -0.1% ( -1% - 1%) 0.761 IntNRQ 480.95 (1.5%) 480.96 (1.8%) 0.0% ( -3% - 3%) 0.997 HighTermMonthSort 1367.53 (2.9%) 1367.57 (2.5%) 0.0% ( -5% - 5%) 0.997 PKLookup 200.22 (2.4%) 200.26 (2.2%) 0.0% ( -4% - 4%) 0.980 AndHighMedDayTaxoFacets 37.34 (1.3%) 37.35 (1.3%) 0.0% ( -2% - 2%) 0.950 BrowseDateSSDVFacets 0.89 (8.9%) 0.89 (9.0%) 0.1% ( -16% - 19%) 0.986 BrowseRandomLabelTaxoFacets 2.42 (6.7%) 2.42 (6.3%) 0.1% ( -12% - 14%) 0.944 HighTermDayOfYearSort 337.48 (1.4%) 337.97 (2.4%) 0.1% ( -3% - 4%) 0.815 OrHighLow 858.90 (2.8%) 860.97 (1.5%) 0.2% ( -3% - 4%) 0.733 Fuzzy2 62.96 (2.4%) 63.12 (2.6%) 0.3% ( -4% - 5%) 0.751 HighTermTitleBDVSort 59.22 (1.2%) 59.40 (1.3%) 0.3% ( -2% - 2%) 0.424 BrowseRandomLabelSSDVFacets 3.26 (12.3%) 3.28 (11.0%) 0.4% ( -20% - 26%) 0.920 Fuzzy1 76.09 (2.4%) 76.38 (2.8%) 0.4% ( -4% - 5%) 0.645 LowSpanNear 23.34 (1.6%) 23.44 (1.1%) 0.4% ( -2% - 3%) 0.352 AndHighHigh 246.95 (4.4%) 247.99 (2.6%) 0.4% ( -6% - 7%) 0.712 Respell 32.82 (1.7%) 32.96 (1.9%) 0.4% ( -3% - 4%) 0.460 AndHighMed 779.67 (1.7%) 783.93 (2.2%) 0.5% ( -3% - 4%) 0.378 OrHighMed 614.98 (4.2%) 618.45 (1.8%) 0.6% ( -5% - 6%) 0.578 MedSpanNear 81.03 (2.1%) 81.50 (1.3%) 0.6% ( -2% - 4%) 0.299 HighSpanNear 69.20 (2.8%) 69.71 (2.2%) 0.7% ( -4% - 5%) 0.363 range 2080.15 (5.2%) 2100.67 (5.0%) 1.0% ( -8% - 11%) 0.541 LowTerm 1783.95 (7.3%) 1802.69 (3.8%) 1.1% ( -9% - 13%) 0.568 BrowseDayOfYearTaxoFacets 3.23 (8.2%) 3.27 (10.5%) 1.3% ( -15% - 21%) 0.651 BrowseDateTaxoFacets 3.21 (8.4%) 3.26 (11.3%) 1.4% ( -16% - 23%) 0.669 OrHighHigh 258.98 (5.9%) 263.06 (3.7%) 1.6% ( -7% - 11%) 0.309 BrowseMonthSSDVFacets 4.41 (6.5%) 4.56 (13.1%) 3.2% ( -15% - 24%) 0.320 BrowseDayOfYearSSDVFacets 4.37 (8.4%) 4.56 (12.1%) 4.2% ( -15% - 26%) 0.201 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
