kkewwei commented on PR #16089: URL: https://github.com/apache/lucene/pull/16089#issuecomment-5249586025
@epotyom Thanks for the detailed explanation. I added a `SubHalfIntNRQ` benchmark task set in `luceneutil`(I'd be happy to submit a PR if necessary.) https://github.com/kkewwei/luceneutil/commit/2c44e536d3a42b686ed4c77e3c10be0976291bfd The benchmark command was: ``` python3 util/src/python/localrun.py wikimediumall ``` After 19 benchmark rounds, the final results are: ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value HighTermTitleBDVSort 3.72 (10.6%) 3.62 (11.0%) -2.9% ( -22% - 21%) 0.402 BrowseDateSSDVFacets 0.69 (11.5%) 0.68 (12.3%) -1.5% ( -22% - 25%) 0.692 OrHighNotHigh 110.14 (4.2%) 108.66 (6.5%) -1.3% ( -11% - 9%) 0.434 OrNotHighHigh 93.69 (4.9%) 92.89 (7.8%) -0.9% ( -12% - 12%) 0.679 AndHighHigh 29.45 (4.5%) 29.22 (5.4%) -0.8% ( -10% - 9%) 0.614 HighTermTitleSort 60.95 (4.1%) 60.55 (6.1%) -0.7% ( -10% - 9%) 0.692 OrHighNotMed 172.16 (5.2%) 171.17 (5.9%) -0.6% ( -11% - 11%) 0.743 AndHighMed 106.80 (3.3%) 106.30 (5.1%) -0.5% ( -8% - 8%) 0.729 IntSet 224.19 (6.4%) 223.60 (6.2%) -0.3% ( -12% - 13%) 0.895 BM25MSM2 2.90 (5.0%) 2.90 (3.4%) -0.1% ( -8% - 8%) 0.935 HighSloppyPhrase 2.08 (6.8%) 2.08 (4.8%) -0.0% ( -10% - 12%) 0.992 MedSloppyPhrase 5.46 (4.4%) 5.46 (8.0%) 0.0% ( -11% - 12%) 0.991 AndHighMedDayTaxoFacets 6.38 (4.4%) 6.38 (5.6%) 0.0% ( -9% - 10%) 0.984 Fuzzy2 38.27 (3.4%) 38.29 (4.2%) 0.1% ( -7% - 7%) 0.962 HighSpanNear 2.56 (5.0%) 2.57 (3.6%) 0.2% ( -7% - 9%) 0.912 OrHighNotLow 299.15 (5.9%) 299.71 (6.9%) 0.2% ( -11% - 13%) 0.927 Fuzzy1 47.54 (5.0%) 47.74 (5.1%) 0.4% ( -9% - 11%) 0.790 MedSpanNear 2.75 (4.3%) 2.76 (4.0%) 0.5% ( -7% - 9%) 0.680 HighTerm 314.23 (3.3%) 316.03 (2.6%) 0.6% ( -5% - 6%) 0.543 IntNRQ 26.50 (4.6%) 26.65 (5.6%) 0.6% ( -9% - 11%) 0.721 LowIntervalsOrdered 40.82 (8.9%) 41.06 (8.9%) 0.6% ( -15% - 20%) 0.835 MedTerm 470.30 (4.6%) 473.25 (3.9%) 0.6% ( -7% - 9%) 0.642 AndMissingHigh 1353.29 (6.8%) 1361.82 (6.6%) 0.6% ( -11% - 15%) 0.766 OrNotHighLow 276.98 (7.1%) 278.93 (7.2%) 0.7% ( -12% - 16%) 0.755 AndHighLow 478.43 (6.8%) 482.88 (7.4%) 0.9% ( -12% - 16%) 0.680 BrowseRandomLabelSSDVFacets 1.83 (7.9%) 1.85 (5.8%) 0.9% ( -11% - 15%) 0.669 LowPhrase 17.35 (3.8%) 17.52 (5.6%) 0.9% ( -8% - 10%) 0.529 OrHighLow 274.67 (2.9%) 277.54 (5.5%) 1.0% ( -7% - 9%) 0.451 PKLookup 172.63 (8.8%) 174.54 (7.8%) 1.1% ( -14% - 19%) 0.673 OrHighMed 100.26 (5.3%) 101.39 (3.9%) 1.1% ( -7% - 10%) 0.441 LowSpanNear 15.76 (3.4%) 15.95 (3.8%) 1.2% ( -5% - 8%) 0.304 OrHighHigh 34.17 (6.6%) 34.59 (4.7%) 1.2% ( -9% - 13%) 0.496 OrNotHighMed 135.60 (5.1%) 137.28 (8.3%) 1.2% ( -11% - 15%) 0.568 HighIntervalsOrdered 7.98 (6.5%) 8.08 (6.0%) 1.3% ( -10% - 14%) 0.511 Wildcard 36.12 (5.2%) 36.62 (4.2%) 1.4% ( -7% - 11%) 0.358 TermDTSort 62.91 (12.0%) 63.77 (11.9%) 1.4% ( -20% - 28%) 0.716 Respell 30.21 (5.2%) 30.63 (4.0%) 1.4% ( -7% - 11%) 0.347 MedTermDayTaxoFacets 7.00 (5.8%) 7.10 (4.5%) 1.5% ( -8% - 12%) 0.368 HighTermMonthSort 1105.13 (6.5%) 1122.67 (8.9%) 1.6% ( -12% - 18%) 0.518 LowSloppyPhrase 31.88 (4.1%) 32.42 (5.0%) 1.7% ( -7% - 11%) 0.239 MedPhrase 89.99 (4.1%) 91.60 (4.9%) 1.8% ( -6% - 11%) 0.213 MedIntervalsOrdered 16.22 (9.1%) 16.51 (5.3%) 1.8% ( -11% - 17%) 0.444 ConstMSM2 9.94 (6.7%) 10.16 (3.6%) 2.2% ( -7% - 13%) 0.204 HighPhrase 31.51 (5.7%) 32.20 (5.2%) 2.2% ( -8% - 13%) 0.206 HighTermDayOfYearSort 150.79 (6.7%) 154.30 (7.3%) 2.3% ( -10% - 17%) 0.291 LowTerm 821.14 (5.6%) 840.80 (6.5%) 2.4% ( -9% - 15%) 0.214 Prefix3 75.96 (7.2%) 78.10 (6.6%) 2.8% ( -10% - 17%) 0.197 BrowseDayOfYearTaxoFacets 3.87 (32.9%) 3.98 (38.4%) 2.8% ( -51% - 110%) 0.802 AndHighHighDayTaxoFacets 4.78 (5.5%) 4.93 (4.5%) 3.2% ( -6% - 13%) 0.045 OrHighMedDayTaxoFacets 0.75 (6.4%) 0.78 (5.8%) 3.7% ( -8% - 16%) 0.058 BrowseMonthSSDVFacets 3.43 (12.3%) 3.57 (21.8%) 3.9% ( -26% - 43%) 0.481 range 2326.07 (7.8%) 2429.10 (11.6%) 4.4% ( -13% - 25%) 0.156 BrowseDayOfYearSSDVFacets 3.20 (14.9%) 3.38 (20.2%) 5.5% ( -25% - 47%) 0.331 BrowseRandomLabelTaxoFacets 2.32 (19.0%) 2.45 (32.8%) 5.6% ( -38% - 70%) 0.509 SubHalfIntNRQ 92.60 (3.8%) 99.04 (7.8%) 7.0% ( -4% - 19%) 0.000 BrowseDateTaxoFacets 3.66 (30.3%) 3.92 (41.0%) 7.2% ( -49% - 112%) 0.530 BrowseMonthTaxoFacets 4.14 (46.8%) 5.04 (63.0%) 21.6% ( -60% - 247%) 0.218 ``` So this change shows about a 7% QPS improvement for the new `SubHalfIntNRQ` workload, which is intended to exercise 1D `IntPoint` range queries that are wide enough to trigger `DocIdSetBuilder.expectMore(cost)`, while still staying below the inverse-query threshold. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
