[
https://issues.apache.org/jira/browse/LUCENE-10334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464619#comment-17464619
]
Feng Guo commented on LUCENE-10334:
-----------------------------------
I'm so sorry to say that there is something wrong with my benchmark: The
localrun script is still using the facets described in luceneutil
[readme|https://github.com/mikemccand/luceneutil/blob/master/README.md], like
this:
{code:python}
facets = (('taxonomy:Date', 'Date'),('sortedset:Month',
'Month'),('sortedset:DayOfYear', 'DayOfYear'))
index = comp.newIndex('lucene_baseline', sourceData, facets=facets,
indexSort='dayOfYearNumericDV:long')
{code}
And i got the result mentioned above with this facets. But when i'm cloning a
new luceneutil and rerun the setup.py, it becomes:
{code:python}
index = comp.newIndex('lucene_baseline', sourceData,
facets = (('taxonomy:Date', 'Date'),
('taxonomy:Month', 'Month'),
('taxonomy:DayOfYear', 'DayOfYear'),
('sortedset:Month', 'Month'),
('sortedset:DayOfYear', 'DayOfYear'),
('taxonomy:RandomLabel', 'RandomLabel'),
('sortedset:RandomLabel', 'RandomLabel')))
{code}
And the result is totally different with this:
{code:java}
TaskQPS baseline StdDevQPS my_modified_version
StdDev Pct diff p-value
BrowseDayOfYearTaxoFacets 13.65 (8.9%) 10.49
(2.6%) -23.2% ( -31% - -12%) 0.000
BrowseMonthTaxoFacets 13.54 (14.6%) 10.89
(2.9%) -19.6% ( -32% - -2%) 0.000
BrowseDateTaxoFacets 13.50 (8.8%) 11.11
(3.7%) -17.7% ( -27% - -5%) 0.000
BrowseRandomLabelTaxoFacets 11.78 (7.0%) 9.94
(5.1%) -15.6% ( -25% - -3%) 0.000
MedTermDayTaxoFacets 47.49 (2.4%) 41.45
(3.4%) -12.7% ( -18% - -7%) 0.000
AndHighMedDayTaxoFacets 130.24 (2.7%) 119.48
(3.9%) -8.3% ( -14% - -1%) 0.000
AndHighHighDayTaxoFacets 28.80 (2.8%) 27.09
(3.1%) -5.9% ( -11% - 0%) 0.000
OrHighMedDayTaxoFacets 9.68 (2.7%) 9.35
(2.8%) -3.4% ( -8% - 2%) 0.000
HighTermDayOfYearSort 139.73 (9.6%) 135.74
(10.2%) -2.9% ( -20% - 18%) 0.361
TermDTSort 151.46 (9.0%) 147.40
(7.7%) -2.7% ( -17% - 15%) 0.311
Fuzzy2 35.22 (6.3%) 34.38
(5.9%) -2.4% ( -13% - 10%) 0.213
MedSloppyPhrase 78.99 (6.7%) 77.21
(7.1%) -2.3% ( -15% - 12%) 0.300
LowTerm 1636.38 (6.4%) 1600.26
(9.6%) -2.2% ( -17% - 14%) 0.392
LowPhrase 252.68 (3.8%) 247.11
(6.5%) -2.2% ( -12% - 8%) 0.189
Respell 61.23 (2.3%) 59.89
(5.0%) -2.2% ( -9% - 5%) 0.078
AndHighHigh 56.54 (2.6%) 55.43
(4.3%) -2.0% ( -8% - 5%) 0.084
MedSpanNear 99.37 (2.4%) 97.44
(5.2%) -1.9% ( -9% - 5%) 0.128
HighSloppyPhrase 28.58 (5.4%) 28.05
(5.4%) -1.8% ( -11% - 9%) 0.280
PKLookup 198.95 (3.0%) 195.34
(4.8%) -1.8% ( -9% - 6%) 0.148
AndHighMed 116.50 (3.3%) 114.65
(4.5%) -1.6% ( -9% - 6%) 0.204
Fuzzy1 75.07 (6.4%) 73.99
(8.1%) -1.4% ( -14% - 13%) 0.532
HighSpanNear 10.73 (2.8%) 10.58
(3.9%) -1.4% ( -7% - 5%) 0.180
LowSpanNear 43.92 (2.4%) 43.30
(3.4%) -1.4% ( -6% - 4%) 0.128
LowSloppyPhrase 14.70 (4.4%) 14.50
(4.2%) -1.3% ( -9% - 7%) 0.329
HighTermMonthSort 148.80 (8.3%) 146.84
(8.1%) -1.3% ( -16% - 16%) 0.612
OrHighMed 103.00 (3.2%) 101.67
(5.1%) -1.3% ( -9% - 7%) 0.341
MedIntervalsOrdered 5.44 (2.5%) 5.37
(2.2%) -1.3% ( -5% - 3%) 0.092
OrHighNotHigh 648.74 (6.7%) 640.81
(8.8%) -1.2% ( -15% - 15%) 0.621
MedPhrase 80.35 (2.7%) 79.38
(4.8%) -1.2% ( -8% - 6%) 0.327
HighTerm 1384.91 (6.8%) 1369.27
(8.8%) -1.1% ( -15% - 15%) 0.650
IntNRQ 127.36 (2.8%) 125.95
(5.8%) -1.1% ( -9% - 7%) 0.440
HighIntervalsOrdered 12.81 (2.6%) 12.67
(2.6%) -1.1% ( -6% - 4%) 0.190
OrHighHigh 28.28 (1.9%) 28.03
(2.6%) -0.9% ( -5% - 3%) 0.218
Wildcard 85.94 (3.2%) 85.20
(3.7%) -0.9% ( -7% - 6%) 0.438
AndHighLow 577.83 (6.0%) 573.01
(6.2%) -0.8% ( -12% - 12%) 0.663
LowIntervalsOrdered 14.53 (2.5%) 14.44
(1.9%) -0.6% ( -4% - 3%) 0.354
OrNotHighHigh 678.90 (6.6%) 675.09
(9.3%) -0.6% ( -15% - 16%) 0.826
HighTermTitleBDVSort 145.07 (10.1%) 144.63
(11.7%) -0.3% ( -20% - 23%) 0.931
Prefix3 136.65 (4.5%) 136.49
(5.1%) -0.1% ( -9% - 9%) 0.937
HighPhrase 27.41 (2.7%) 27.45
(4.4%) 0.1% ( -6% - 7%) 0.898
MedTerm 1470.28 (5.9%) 1472.69
(8.1%) 0.2% ( -13% - 15%) 0.941
OrHighNotMed 596.47 (4.9%) 597.84
(8.0%) 0.2% ( -12% - 13%) 0.914
OrHighLow 419.76 (6.5%) 423.67
(7.7%) 0.9% ( -12% - 16%) 0.679
OrHighNotLow 665.70 (7.9%) 672.36
(7.2%) 1.0% ( -13% - 17%) 0.675
OrNotHighLow 698.46 (3.5%) 706.62
(8.8%) 1.2% ( -10% - 13%) 0.581
OrNotHighMed 586.16 (6.9%) 593.42
(10.1%) 1.2% ( -14% - 19%) 0.651
BrowseRandomLabelSSDVFacets 10.15 (3.3%) 17.13
(8.3%) 68.8% ( 55% - 83%) 0.000
BrowseMonthSSDVFacets 15.36 (3.6%) 32.91
(19.2%) 114.3% ( 88% - 142%) 0.000
BrowseDayOfYearSSDVFacets 14.02 (3.1%) 32.86
(16.2%) 134.4% ( 111% - 158%) 0.000
{code}
This is something expected now... I'll try some other patches to try to solve
this but i'd say so sorry again for the noise here!
> Introduce a BlockReader based on ForUtil and use it for NumericDocValues
> ------------------------------------------------------------------------
>
> Key: LUCENE-10334
> URL: https://issues.apache.org/jira/browse/LUCENE-10334
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/codecs
> Reporter: Feng Guo
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Previous talk is here: https://github.com/apache/lucene/pull/557
> This is trying to add a new BlockReader based on ForUtil to replace the
> DirectReader we are using for NumericDocvalues
> *Benchmark based on wiki10m*
> {code:java}
> TaskQPS baseline StdDevQPS
> my_modified_version StdDev Pct diff p-value
> OrNotHighHigh 694.17 (8.2%) 685.83
> (7.0%) -1.2% ( -15% - 15%) 0.618
> Respell 75.15 (2.7%) 74.32
> (2.0%) -1.1% ( -5% - 3%) 0.146
> Prefix3 220.11 (5.1%) 217.78
> (5.8%) -1.1% ( -11% - 10%) 0.541
> Wildcard 129.75 (3.7%) 128.63
> (2.5%) -0.9% ( -6% - 5%) 0.383
> LowSpanNear 68.54 (2.1%) 68.00
> (2.4%) -0.8% ( -5% - 3%) 0.269
> OrNotHighMed 732.90 (6.8%) 727.49
> (5.3%) -0.7% ( -12% - 12%) 0.703
> BrowseRandomLabelTaxoFacets 11879.03 (8.6%) 11799.33
> (5.5%) -0.7% ( -13% - 14%) 0.769
> HighSloppyPhrase 6.87 (2.9%) 6.83
> (2.3%) -0.6% ( -5% - 4%) 0.496
> OrHighNotMed 827.54 (9.2%) 822.94
> (8.0%) -0.6% ( -16% - 18%) 0.838
> MedSpanNear 18.92 (5.7%) 18.82
> (5.6%) -0.5% ( -11% - 11%) 0.759
> OrHighMedDayTaxoFacets 10.27 (4.0%) 10.21
> (4.3%) -0.5% ( -8% - 8%) 0.676
> PKLookup 207.98 (4.0%) 206.85
> (2.7%) -0.5% ( -7% - 6%) 0.621
> LowIntervalsOrdered 159.17 (2.3%) 158.32
> (2.2%) -0.5% ( -4% - 3%) 0.445
> HighSpanNear 6.32 (4.2%) 6.28
> (4.1%) -0.5% ( -8% - 8%) 0.691
> MedIntervalsOrdered 85.31 (3.2%) 84.88
> (2.9%) -0.5% ( -6% - 5%) 0.607
> HighTerm 1170.55 (5.8%) 1164.79
> (3.9%) -0.5% ( -9% - 9%) 0.753
> LowSloppyPhrase 14.54 (3.1%) 14.48
> (2.9%) -0.4% ( -6% - 5%) 0.651
> HighPhrase 112.81 (4.4%) 112.39
> (4.1%) -0.4% ( -8% - 8%) 0.781
> OrNotHighLow 858.02 (5.9%) 854.99
> (4.8%) -0.4% ( -10% - 10%) 0.835
> HighIntervalsOrdered 25.08 (2.8%) 25.00
> (2.6%) -0.3% ( -5% - 5%) 0.701
> MedPhrase 27.20 (2.1%) 27.11
> (2.9%) -0.3% ( -5% - 4%) 0.689
> MedTermDayTaxoFacets 81.55 (2.3%) 81.35
> (2.9%) -0.3% ( -5% - 5%) 0.762
> IntNRQ 63.36 (2.0%) 63.21
> (2.5%) -0.2% ( -4% - 4%) 0.740
> Fuzzy2 73.24 (5.5%) 73.10
> (6.2%) -0.2% ( -11% - 12%) 0.916
> AndHighMedDayTaxoFacets 76.08 (3.5%) 75.98
> (3.4%) -0.1% ( -6% - 7%) 0.905
> AndHighHigh 62.20 (2.0%) 62.18
> (2.4%) -0.0% ( -4% - 4%) 0.954
> BrowseMonthTaxoFacets 11993.48 (6.7%) 11989.53
> (4.8%) -0.0% ( -10% - 12%) 0.986
> OrHighNotLow 732.82 (7.2%) 732.80
> (6.2%) -0.0% ( -12% - 14%) 0.999
> Fuzzy1 46.43 (5.3%) 46.45
> (6.0%) 0.0% ( -10% - 11%) 0.989
> LowTerm 1608.25 (6.0%) 1608.84
> (4.9%) 0.0% ( -10% - 11%) 0.983
> OrHighMed 75.90 (2.3%) 75.93
> (1.8%) 0.0% ( -3% - 4%) 0.939
> LowPhrase 273.81 (2.9%) 274.04
> (3.3%) 0.1% ( -5% - 6%) 0.932
> AndHighLow 717.24 (6.1%) 718.17
> (3.3%) 0.1% ( -8% - 10%) 0.933
> AndHighHighDayTaxoFacets 39.63 (2.5%) 39.69
> (2.6%) 0.1% ( -4% - 5%) 0.862
> OrHighHigh 34.63 (1.8%) 34.68
> (2.0%) 0.1% ( -3% - 4%) 0.821
> MedSloppyPhrase 158.80 (2.8%) 159.09
> (2.6%) 0.2% ( -5% - 5%) 0.832
> OrHighLow 257.77 (2.9%) 258.46
> (4.6%) 0.3% ( -7% - 8%) 0.826
> AndHighMed 133.43 (2.1%) 133.79
> (2.7%) 0.3% ( -4% - 5%) 0.726
> HighTermMonthSort 145.28 (10.8%) 145.88
> (11.2%) 0.4% ( -19% - 25%) 0.905
> OrHighNotHigh 834.99 (6.1%) 839.62
> (5.7%) 0.6% ( -10% - 13%) 0.766
> TermDTSort 83.66 (9.6%) 84.30
> (11.1%) 0.8% ( -18% - 23%) 0.817
> BrowseDayOfYearTaxoFacets 11639.59 (5.1%) 11777.38
> (6.0%) 1.2% ( -9% - 12%) 0.502
> MedTerm 1473.62 (7.4%) 1493.79
> (6.4%) 1.4% ( -11% - 16%) 0.530
> HighTermTitleBDVSort 114.98 (16.7%) 117.30
> (18.8%) 2.0% ( -28% - 45%) 0.720
> HighTermDayOfYearSort 128.29 (17.2%) 132.83
> (22.6%) 3.5% ( -30% - 52%) 0.577
> BrowseDateTaxoFacets 19.25 (20.4%) 26.77
> (3.7%) 39.1% ( 12% - 79%) 0.000
> BrowseRandomLabelSSDVFacets 10.38 (3.5%) 18.03
> (6.8%) 73.7% ( 61% - 87%) 0.000
> BrowseMonthSSDVFacets 15.71 (3.6%) 34.59
> (12.4%) 120.1% ( 100% - 141%) 0.000
> BrowseDayOfYearSSDVFacets 14.31 (3.3%) 33.54
> (12.9%) 134.4% ( 114% - 155%) 0.000
> {code}
> *candidate*
> {code:java}
> PERCENT CPU SAMPLES STACK
> 3.48% 9280
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$BlockImpactsPostingsEnum#advance()
> 3.41% 9082
> org.apache.lucene.facet.sortedset.SortedSetDocValuesFacetCounts#countOneSegment()
> 3.32% 8836
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$EverythingEnum#nextPosition()
> 2.72% 7260
> org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$20#ordValue()
> 2.03% 5423
> org.apache.lucene.queries.spans.NearSpansOrdered#stretchToOrder()
> 1.91% 5094
> org.apache.lucene.queries.spans.TermSpans#nextStartPosition()
> 1.90% 5063
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$EverythingEnum#advance()
> 1.80% 4787
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader#findFirstGreater()
> 1.72% 4574 org.apache.lucene.search.PhraseScorer$1#matches()
> 1.55% 4141 org.apache.lucene.util.PriorityQueue#upHeap()
> 1.55% 4141
> org.apache.lucene.codecs.lucene90.ForUtil#expand8()
> 1.53% 4073
> org.apache.lucene.queries.spans.NearSpansOrdered#nextStartPosition()
> 1.47% 3929 org.apache.lucene.util.packed.BlockReader#get()
> 1.39% 3703 org.apache.lucene.search.ConjunctionDISI#doNext()
> 1.35% 3593
> org.apache.lucene.codecs.lucene90.Lucene90NormsProducer$3#longValue()
> 1.32% 3514
> jdk.internal.misc.ScopedMemoryAccess#getByteInternal()
> 1.21% 3236
> org.apache.lucene.search.similarities.BM25Similarity$BM25Scorer#score()
> 1.13% 3003
> org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$20#nextDoc()
> 1.05% 2808
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$BlockImpactsPostingsEnum#nextPosition()
> 1.04% 2780
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$EverythingEnum#skipPositions()
> 1.03% 2750
> org.apache.lucene.search.BooleanScorer$OrCollector#collect()
> 1.03% 2732
> org.apache.lucene.search.SloppyPhraseMatcher#maxFreq()
> 0.99% 2627
> org.apache.lucene.queries.intervals.OrderedIntervalsSource$OrderedIntervalIterator#nextInterval()
> 0.98% 2610
> org.apache.lucene.search.MultiCollector$MultiLeafCollector#collect()
> 0.89% 2368
> org.apache.lucene.search.SloppyPhraseMatcher#initSimple()
> 0.88% 2350
> org.apache.lucene.queries.spans.NearSpansOrdered#advancePosition()
> 0.87% 2312
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$BlockDocsEnum#advance()
> 0.84% 2252 org.apache.lucene.util.PriorityQueue#add()
> 0.82% 2176
> org.apache.lucene.queries.spans.SpanScorer#setFreqCurrentDoc()
> 0.81% 2161
> org.apache.lucene.codecs.lucene90.PForUtil#decode()
> {code}
> *baseline*
> {code:java}
> PERCENT CPU SAMPLES STACK
> 4.22% 12298
> org.apache.lucene.facet.sortedset.SortedSetDocValuesFacetCounts#countOneSegment()
> 3.25% 9468
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$BlockImpactsPostingsEnum#advance()
> 3.04% 8872
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$EverythingEnum#nextPosition()
> 2.26% 6576
> org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$20#ordValue()
> 2.06% 5993
> org.apache.lucene.util.packed.DirectReader$DirectPackedReader12#get()
> 1.90% 5528
> org.apache.lucene.queries.spans.NearSpansOrdered#stretchToOrder()
> 1.81% 5266
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$EverythingEnum#advance()
> 1.75% 5116
> org.apache.lucene.queries.spans.TermSpans#nextStartPosition()
> 1.53% 4469
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader#findFirstGreater()
> 1.51% 4392 org.apache.lucene.search.PhraseScorer$1#matches()
> 1.44% 4204 org.apache.lucene.util.PriorityQueue#upHeap()
> 1.37% 3999
> jdk.internal.misc.ScopedMemoryAccess#getByteInternal()
> 1.37% 3992
> org.apache.lucene.codecs.lucene90.ForUtil#expand8()
> 1.37% 3991
> org.apache.lucene.queries.spans.NearSpansOrdered#nextStartPosition()
> 1.33% 3869 org.apache.lucene.search.ConjunctionDISI#doNext()
> 1.27% 3688
> org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$20#nextDoc()
> 1.24% 3606 java.nio.Buffer#scope()
> 1.23% 3593
> org.apache.lucene.codecs.lucene90.Lucene90NormsProducer$3#longValue()
> 1.20% 3491
> org.apache.lucene.util.packed.DirectReader$DirectPackedReader20#get()
> 1.16% 3392 java.nio.Buffer#checkIndex()
> 1.09% 3186
> org.apache.lucene.util.packed.DirectReader$DirectPackedReader4#get()
> 1.09% 3164
> org.apache.lucene.search.similarities.BM25Similarity$BM25Scorer#score()
> 1.01% 2946
> org.apache.lucene.store.ByteBufferGuard#ensureValid()
> 0.95% 2772
> org.apache.lucene.search.BooleanScorer$OrCollector#collect()
> 0.95% 2766
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$EverythingEnum#skipPositions()
> 0.95% 2763
> org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$BlockImpactsPostingsEnum#nextPosition()
> 0.93% 2699
> org.apache.lucene.search.SloppyPhraseMatcher#maxFreq()
> 0.92% 2678
> org.apache.lucene.search.MultiCollector$MultiLeafCollector#collect()
> 0.87% 2545
> org.apache.lucene.queries.intervals.OrderedIntervalsSource$OrderedIntervalIterator#nextInterval()
> 0.85% 2479
> org.apache.lucene.search.SloppyPhraseMatcher#initSimple()
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]