[
https://issues.apache.org/jira/browse/LUCENE-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475438#comment-17475438
]
Greg Miller commented on LUCENE-10374:
--------------------------------------
It appears that LUCENE-10350 was responsible for the nightly benchmark
regressions but I can't reason about how it would cause a regression. Both
[~gf2121] and I saw significant performance improvements associated with these
same benchmark tasks when running locally (results below; and note that this
benchmark is "reversed" in that it was trying to measure the impact of
reverting LUCENE-10350... so the baseline has the change and the candidate
reverts it). So it remains a mystery why the discrepancy would exist.
{code:java}
TaskQPS baseline StdDevQPS candidate
StdDev Pct diff p-value
BrowseDayOfYearTaxoFacets 30.81 (24.0%) 14.56
(4.5%) -52.7% ( -65% - -31%) 0.000
BrowseDateTaxoFacets 30.38 (23.9%) 14.52
(4.5%) -52.2% ( -65% - -31%) 0.000
BrowseMonthTaxoFacets 30.08 (20.9%) 15.73
(3.9%) -47.7% ( -59% - -29%) 0.000
BrowseRandomLabelTaxoFacets 23.21 (24.6%) 12.43
(4.4%) -46.4% ( -60% - -23%) 0.000
MedTermDayTaxoFacets 34.20 (4.5%) 33.65
(4.8%) -1.6% ( -10% - 7%) 0.273
TermDTSort 185.78 (20.4%) 183.85
(20.8%) -1.0% ( -35% - 50%) 0.873
AndHighHigh 53.56 (3.3%) 53.04
(3.3%) -1.0% ( -7% - 5%) 0.361
LowPhrase 97.47 (3.4%) 97.06
(2.1%) -0.4% ( -5% - 5%) 0.634
HighSpanNear 8.59 (4.2%) 8.55
(4.4%) -0.4% ( -8% - 8%) 0.761
OrHighLow 668.71 (1.6%) 666.14
(2.4%) -0.4% ( -4% - 3%) 0.546
AndHighMed 227.67 (1.8%) 226.80
(2.1%) -0.4% ( -4% - 3%) 0.533
OrHighMedDayTaxoFacets 7.26 (5.2%) 7.24
(5.8%) -0.4% ( -10% - 11%) 0.832
OrHighMed 117.01 (3.7%) 116.66
(2.9%) -0.3% ( -6% - 6%) 0.774
MedSpanNear 49.68 (4.0%) 49.55
(4.4%) -0.3% ( -8% - 8%) 0.847
LowSpanNear 45.83 (3.1%) 45.72
(2.7%) -0.2% ( -5% - 5%) 0.796
MedPhrase 95.22 (3.2%) 95.12
(2.2%) -0.1% ( -5% - 5%) 0.897
OrHighHigh 36.26 (3.7%) 36.22
(3.2%) -0.1% ( -6% - 7%) 0.918
IntNRQ 97.48 (1.4%) 97.57
(1.1%) 0.1% ( -2% - 2%) 0.822
MedSloppyPhrase 28.73 (2.4%) 28.76
(2.8%) 0.1% ( -4% - 5%) 0.912
BrowseRandomLabelSSDVFacets 9.48 (3.8%) 9.49
(3.5%) 0.1% ( -6% - 7%) 0.929
OrNotHighHigh 884.42 (3.0%) 885.49
(3.2%) 0.1% ( -5% - 6%) 0.902
LowIntervalsOrdered 90.92 (4.1%) 91.10
(3.9%) 0.2% ( -7% - 8%) 0.878
OrNotHighMed 1089.50 (2.2%) 1091.92
(3.0%) 0.2% ( -4% - 5%) 0.788
OrHighNotHigh 826.99 (3.7%) 829.46
(3.3%) 0.3% ( -6% - 7%) 0.787
HighIntervalsOrdered 7.48 (6.8%) 7.50
(6.5%) 0.3% ( -12% - 14%) 0.885
MedTerm 1890.85 (2.9%) 1896.72
(2.6%) 0.3% ( -5% - 5%) 0.721
MedIntervalsOrdered 6.54 (4.5%) 6.57
(4.2%) 0.4% ( -7% - 9%) 0.770
AndHighHighDayTaxoFacets 15.93 (3.1%) 16.00
(3.0%) 0.4% ( -5% - 6%) 0.662
LowTerm 1977.37 (2.7%) 1986.22
(3.5%) 0.4% ( -5% - 6%) 0.648
LowSloppyPhrase 86.15 (3.9%) 86.56
(4.4%) 0.5% ( -7% - 9%) 0.720
HighTerm 1494.75 (3.1%) 1501.82
(2.9%) 0.5% ( -5% - 6%) 0.622
Fuzzy2 67.47 (2.0%) 67.80
(2.1%) 0.5% ( -3% - 4%) 0.450
OrHighNotLow 1388.49 (3.0%) 1395.34
(2.4%) 0.5% ( -4% - 5%) 0.560
AndHighMedDayTaxoFacets 89.88 (2.3%) 90.38
(2.1%) 0.6% ( -3% - 5%) 0.418
BrowseMonthSSDVFacets 14.54 (20.6%) 14.63
(21.8%) 0.6% ( -34% - 54%) 0.927
HighSloppyPhrase 11.53 (4.3%) 11.60
(5.1%) 0.6% ( -8% - 10%) 0.678
HighTermTitleBDVSort 122.18 (14.6%) 122.97
(17.4%) 0.6% ( -27% - 38%) 0.898
HighPhrase 432.20 (3.3%) 435.35
(2.4%) 0.7% ( -4% - 6%) 0.426
Fuzzy1 81.65 (2.1%) 82.25
(2.2%) 0.7% ( -3% - 5%) 0.280
OrHighNotMed 1060.43 (3.0%) 1068.54
(3.0%) 0.8% ( -5% - 6%) 0.422
Respell 63.22 (2.5%) 63.78
(2.4%) 0.9% ( -3% - 5%) 0.255
AndHighLow 1108.01 (2.6%) 1118.78
(3.1%) 1.0% ( -4% - 6%) 0.287
PKLookup 171.13 (3.0%) 173.05
(4.8%) 1.1% ( -6% - 9%) 0.380
HighTermDayOfYearSort 37.22 (25.8%) 37.84
(22.8%) 1.7% ( -37% - 67%) 0.827
Wildcard 85.54 (5.3%) 87.11
(5.0%) 1.8% ( -8% - 12%) 0.262
OrNotHighLow 938.56 (3.1%) 955.98
(3.0%) 1.9% ( -4% - 8%) 0.054
Prefix3 146.06 (9.7%) 150.33
(8.5%) 2.9% ( -13% - 23%) 0.307
HighTermMonthSort 123.10 (14.9%) 126.94
(16.8%) 3.1% ( -24% - 40%) 0.534
BrowseDayOfYearSSDVFacets 12.11 (9.5%) 12.56
(12.9%) 3.7% ( -17% - 28%) 0.297{code}
> Track down the "browse" taxonomy faceting qps regression
> --------------------------------------------------------
>
> Key: LUCENE-10374
> URL: https://issues.apache.org/jira/browse/LUCENE-10374
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/facet
> Reporter: Greg Miller
> Priority: Major
>
> We need to track down the source of the regression observed here:
> [https://home.apache.org/~mikemccand/lucenebench/2022.01.10.18.03.12.html.]
> Some details on the regression hunting are in
> [https://github.com/apache/lucene/pull/597.]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
