[ 
https://issues.apache.org/jira/browse/LUCENE-10334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17466709#comment-17466709
 ] 

Feng Guo edited comment on LUCENE-10334 at 12/30/21, 10:06 AM:
---------------------------------------------------------------

In order to save reading time, I deleted some previous progress comments and 
try to make a final summary here.

??one idea is we could try using the new block compression just for ordinals as 
a start??

Thanks [~rcmuir] for the suggestion! I made some optimizations in this approach 
and browse taxo tasks (Browse*TaxoFacets) are getting speed up too. So the 
benchmark is telling "dense faster sparse slower" instead of "SSDV faster Taxos 
slower" now. I suspect we probably did not see a SSDV regression just because 
we have not added reading sparse SSDV values tasks, e.g. a 
{{{}MedTermDaySSDVFacets{}}}.

I've got two schemes in mind so far:

*ForUtil Approach*

This approach tends to make file format friendly to block decoding and decode 
block based on the efficient ForUtil (with SIMD opt) for each get. As a result 
we can get a rather delicious (130%) speed up in {{Browse*}} tasks. But we also 
get a slight (10%) regression in tasks that reading facets with a query (like 
MedTermDayTaxoFacets) since we are reading sparse values there and we need to 
decompress the whole 128 values block even we only need one value in that block.

Here is the [code|https://github.com/apache/lucene/pull/562] and luceneutil 
benchmark:
{code:java}
                            TaskQPS baseline      StdDevQPS my_modified_version 
     StdDev                Pct diff p-value
         AndHighMedDayTaxoFacets       71.49      (2.1%)       64.72      
(2.0%)   -9.5% ( -13% -   -5%) 0.000
            MedTermDayTaxoFacets       25.79      (2.6%)       24.00      
(1.8%)   -6.9% ( -11% -   -2%) 0.000
        AndHighHighDayTaxoFacets       13.13      (3.4%)       12.63      
(3.1%)   -3.9% ( -10% -    2%) 0.000
          OrHighMedDayTaxoFacets       13.71      (4.1%)       13.41      
(4.7%)   -2.2% ( -10% -    6%) 0.118
                        PKLookup      204.87      (3.9%)      203.03      
(3.6%)   -0.9% (  -8% -    6%) 0.450
                         Prefix3      113.85      (3.6%)      113.32      
(4.6%)   -0.5% (  -8% -    8%) 0.724
                    HighSpanNear       25.34      (2.5%)       25.26      
(3.1%)   -0.3% (  -5% -    5%) 0.714
                     LowSpanNear       55.96      (2.0%)       55.80      
(2.1%)   -0.3% (  -4% -    3%) 0.658
                     MedSpanNear       56.84      (2.4%)       56.90      
(2.2%)    0.1% (  -4% -    4%) 0.895
                 MedSloppyPhrase       26.57      (1.8%)       26.60      
(1.9%)    0.1% (  -3% -    3%) 0.831
                HighSloppyPhrase       30.20      (3.7%)       30.24      
(3.6%)    0.2% (  -6% -    7%) 0.890
                       OrHighMed       49.96      (2.1%)       50.06      
(1.7%)    0.2% (  -3% -    4%) 0.742
                      AndHighMed       96.70      (2.9%)       96.95      
(2.6%)    0.3% (  -5% -    5%) 0.772
             LowIntervalsOrdered       23.32      (4.6%)       23.38      
(4.5%)    0.3% (  -8% -    9%) 0.856
                      OrHighHigh       38.09      (1.9%)       38.20      
(1.8%)    0.3% (  -3% -    4%) 0.643
                      TermDTSort      128.55     (14.7%)      128.94     
(11.6%)    0.3% ( -22% -   31%) 0.942
                          Fuzzy1       99.54      (7.1%)       99.86      
(8.0%)    0.3% ( -13% -   16%) 0.893
            HighIntervalsOrdered       15.58      (2.6%)       15.65      
(2.6%)    0.4% (  -4% -    5%) 0.636
                         Respell       63.96      (1.9%)       64.22      
(2.3%)    0.4% (  -3% -    4%) 0.542
                   OrHighNotHigh      611.12      (5.8%)      613.85      
(6.2%)    0.4% ( -10% -   13%) 0.814
             MedIntervalsOrdered       59.48      (5.2%)       59.75      
(5.1%)    0.5% (  -9% -   11%) 0.780
                     AndHighHigh       58.76      (3.0%)       59.16      
(3.0%)    0.7% (  -5% -    6%) 0.478
                   OrNotHighHigh      619.53      (6.0%)      623.79      
(7.1%)    0.7% ( -11% -   14%) 0.740
                      HighPhrase       31.00      (2.5%)       31.26      
(2.7%)    0.8% (  -4% -    6%) 0.307
                      AndHighLow      828.41      (5.9%)      835.65      
(7.1%)    0.9% ( -11% -   14%) 0.672
                    OrNotHighLow      986.46      (6.8%)      995.13     
(10.5%)    0.9% ( -15% -   19%) 0.752
            HighTermTitleBDVSort      110.39     (12.3%)      111.38     
(11.1%)    0.9% ( -20% -   27%) 0.807
                          IntNRQ      151.29      (2.6%)      152.96      
(3.5%)    1.1% (  -4% -    7%) 0.262
                         LowTerm     1876.18      (7.8%)     1897.19      
(8.3%)    1.1% ( -13% -   18%) 0.660
           HighTermDayOfYearSort      108.34     (18.9%)      109.87     
(17.4%)    1.4% ( -29% -   46%) 0.805
               HighTermMonthSort       65.84     (11.0%)       66.78     
(11.7%)    1.4% ( -19% -   27%) 0.689
                    OrHighNotMed      770.05      (5.3%)      782.54      
(8.8%)    1.6% ( -11% -   16%) 0.480
                        Wildcard      182.10      (5.5%)      185.24      
(7.2%)    1.7% ( -10% -   15%) 0.394
                 LowSloppyPhrase       33.75      (6.6%)       34.35      
(8.8%)    1.8% ( -12% -   18%) 0.478
                       MedPhrase      161.57      (3.8%)      164.62      
(6.1%)    1.9% (  -7% -   12%) 0.242
                    OrHighNotLow      679.46      (7.2%)      693.59      
(7.6%)    2.1% ( -11% -   18%) 0.374
                    OrNotHighMed      690.91      (7.4%)      706.15      
(8.8%)    2.2% ( -13% -   19%) 0.390
                        HighTerm     1388.14      (6.3%)     1420.26      
(7.8%)    2.3% ( -11% -   17%) 0.302
                       LowPhrase      410.16      (5.0%)      420.38      
(5.0%)    2.5% (  -7% -   13%) 0.114
                       OrHighLow      479.96      (5.1%)      492.39      
(5.7%)    2.6% (  -7% -   14%) 0.128
                         MedTerm     1575.41      (5.9%)     1618.88      
(8.2%)    2.8% ( -10% -   17%) 0.221
                          Fuzzy2       64.75      (8.3%)       66.76      
(8.3%)    3.1% ( -12% -   21%) 0.237
           BrowseMonthTaxoFacets       14.39     (12.1%)       18.58     
(17.3%)   29.1% (   0% -   66%) 0.000
     BrowseRandomLabelTaxoFacets       12.01      (8.5%)       17.01     
(18.2%)   41.6% (  13% -   74%) 0.000
            BrowseDateTaxoFacets       13.72     (11.2%)       19.83     
(26.5%)   44.5% (   6% -   92%) 0.000
       BrowseDayOfYearTaxoFacets       13.84     (11.5%)       20.03     
(27.4%)   44.8% (   5% -   94%) 0.000
     BrowseRandomLabelSSDVFacets       10.31      (2.6%)       17.72      
(4.2%)   71.9% (  63% -   80%) 0.000
           BrowseMonthSSDVFacets       15.56      (3.3%)       34.58     
(12.3%)  122.3% ( 103% -  142%) 0.000
       BrowseDayOfYearSSDVFacets       14.17      (2.9%)       32.91     
(11.6%)  132.3% ( 114% -  151%) 0.000
{code}
*Detect Warm Approach*

This approach is keeping the origin file format to keep the high efficiency of 
reading sparse values, but try to detect a dense reading and do some block 
decoding for these situations. To be specific, we assume user is reading dense 
values if more than 80% values were required in the first block. And then we 
can use block decoding for following gets. The approach is limited to be used 
in forward reading cases and won't be as efficient as ForUtil in block decoding 
but it is nearly a net win.

Here is the [code|https://github.com/apache/lucene/pull/570] and luceneutil 
benchmark:
{code:java}
                            TaskQPS baseline      StdDevQPS my_modified_version 
     StdDev                Pct diff p-value
                          Fuzzy2       69.41     (10.7%)       67.98     
(12.2%)   -2.1% ( -22% -   23%) 0.572
            MedTermDayTaxoFacets       26.19      (2.6%)       25.81      
(3.0%)   -1.5% (  -6% -    4%) 0.099
          OrHighMedDayTaxoFacets       14.74      (2.6%)       14.53      
(2.3%)   -1.4% (  -6% -    3%) 0.066
         AndHighMedDayTaxoFacets      103.03      (2.2%)      101.62      
(2.7%)   -1.4% (  -6% -    3%) 0.080
            HighTermTitleBDVSort      104.13     (12.4%)      103.17     
(11.2%)   -0.9% ( -21% -   25%) 0.805
           HighTermDayOfYearSort      112.59     (15.5%)      111.69      
(9.4%)   -0.8% ( -22% -   28%) 0.844
                          Fuzzy1       58.74      (3.9%)       58.29      
(4.6%)   -0.8% (  -8% -    8%) 0.566
                 LowSloppyPhrase       79.62      (2.1%)       79.32      
(2.3%)   -0.4% (  -4% -    4%) 0.572
                      AndHighMed      143.05      (2.4%)      142.56      
(3.3%)   -0.3% (  -5% -    5%) 0.709
                HighSloppyPhrase        6.83      (3.6%)        6.81      
(3.3%)   -0.3% (  -6% -    6%) 0.765
                 MedSloppyPhrase       22.60      (3.0%)       22.54      
(2.8%)   -0.3% (  -5% -    5%) 0.775
                        PKLookup      200.00      (2.9%)      199.51      
(2.3%)   -0.2% (  -5% -    5%) 0.769
                        HighTerm     1125.74      (8.0%)     1123.52      
(7.0%)   -0.2% ( -14% -   16%) 0.934
            HighIntervalsOrdered        8.78      (3.0%)        8.77      
(2.4%)   -0.1% (  -5% -    5%) 0.875
             MedIntervalsOrdered       15.26      (3.3%)       15.24      
(2.5%)   -0.1% (  -5% -    5%) 0.895
                        Wildcard       46.32      (3.7%)       46.29      
(3.0%)   -0.1% (  -6% -    6%) 0.949
        AndHighHighDayTaxoFacets       14.24      (3.4%)       14.24      
(3.9%)   -0.0% (  -7% -    7%) 0.997
                         LowTerm     1722.24      (9.2%)     1722.17      
(7.4%)   -0.0% ( -15% -   18%) 0.999
                     AndHighHigh       83.82      (3.8%)       83.87      
(4.4%)    0.1% (  -7% -    8%) 0.961
                     LowSpanNear       15.29      (2.1%)       15.31      
(2.1%)    0.1% (  -3% -    4%) 0.823
                          IntNRQ      131.51      (2.1%)      131.80      
(1.4%)    0.2% (  -3% -    3%) 0.698
                    OrHighNotLow      800.98     (10.3%)      802.86     
(11.7%)    0.2% ( -19% -   24%) 0.946
                    HighSpanNear        6.60      (2.7%)        6.62      
(2.7%)    0.3% (  -4% -    5%) 0.741
                       MedPhrase       34.65      (2.9%)       34.75      
(2.6%)    0.3% (  -5% -    5%) 0.734
             LowIntervalsOrdered      109.61      (3.3%)      110.06      
(3.3%)    0.4% (  -5% -    7%) 0.691
                     MedSpanNear       53.05      (1.7%)       53.28      
(1.7%)    0.4% (  -2% -    3%) 0.426
                      OrHighHigh       33.33      (2.0%)       33.48      
(2.1%)    0.4% (  -3% -    4%) 0.504
                       LowPhrase      542.02      (7.8%)      544.90      
(6.1%)    0.5% ( -12% -   15%) 0.811
                         Respell       58.48      (2.5%)       58.83      
(3.0%)    0.6% (  -4% -    6%) 0.489
               HighTermMonthSort      104.06     (11.1%)      104.71      
(9.2%)    0.6% ( -17% -   23%) 0.847
                         Prefix3      189.59      (5.9%)      190.87      
(6.2%)    0.7% ( -10% -   13%) 0.725
                         MedTerm     1510.36      (7.2%)     1520.89      
(7.4%)    0.7% ( -13% -   16%) 0.763
                       OrHighMed       72.60      (2.6%)       73.19      
(2.8%)    0.8% (  -4% -    6%) 0.341
                      TermDTSort       87.43     (17.9%)       88.58     
(18.8%)    1.3% ( -30% -   46%) 0.821
                      AndHighLow      753.17      (6.4%)      765.05      
(8.0%)    1.6% ( -12% -   17%) 0.490
                   OrHighNotHigh      649.16      (6.5%)      659.67      
(9.4%)    1.6% ( -13% -   18%) 0.525
                    OrNotHighMed      610.85      (6.6%)      623.40      
(8.8%)    2.1% ( -12% -   18%) 0.404
                    OrNotHighLow      809.36      (5.4%)      826.49     
(10.4%)    2.1% ( -12% -   18%) 0.420
                   OrNotHighHigh      600.31      (5.1%)      613.87      
(9.3%)    2.3% ( -11% -   17%) 0.342
                       OrHighLow      451.44      (6.0%)      462.66      
(7.8%)    2.5% ( -10% -   17%) 0.260
                    OrHighNotMed      675.68      (7.6%)      693.79      
(9.8%)    2.7% ( -13% -   21%) 0.332
                      HighPhrase      265.15      (6.5%)      273.05      
(8.3%)    3.0% ( -11% -   19%) 0.208
           BrowseMonthTaxoFacets       14.10     (13.1%)       15.57      
(7.5%)   10.4% (  -8% -   35%) 0.002
     BrowseRandomLabelSSDVFacets       10.25      (3.1%)       12.63      
(4.3%)   23.1% (  15% -   31%) 0.000
     BrowseRandomLabelTaxoFacets       11.84      (8.6%)       14.74      
(8.7%)   24.5% (   6% -   45%) 0.000
            BrowseDateTaxoFacets       13.50     (12.0%)       17.07     
(10.2%)   26.5% (   3% -   55%) 0.000
       BrowseDayOfYearTaxoFacets       13.61     (12.3%)       17.26     
(10.3%)   26.8% (   3% -   56%) 0.000
           BrowseMonthSSDVFacets       15.47      (3.6%)       19.69      
(6.8%)   27.3% (  16% -   39%) 0.000
       BrowseDayOfYearSSDVFacets       14.12      (3.1%)       19.19      
(6.9%)   35.9% (  25% -   47%) 0.000
{code}
I wonder which way you will prefer? or if you want to try some other ideas but 
having no time I'm willing to help too. Any feedback will be welcome :)


was (Author: gf2121):
In order to save reading time, I deleted some previous progress comments and 
try to make a final summary here.

??one idea is we could try using the new block compression just for ordinals as 
a start??

Thanks [~rcmuir] for the suggestion! I made some optimizations in this approach 
and browse taxo tasks (Browse*TaxoFacets) are getting speed up too. So the 
benchmark is telling "dense faster sparse slower" instead of "SSDV faster Taxos 
slower" now. I suspect we probably did not see a SSDV regression just because 
we have not added reading sparse SSDV values tasks, e.g. a 
{{{}MedTermDaySSDVFacets{}}}.

I've got two schemes in mind so far:

*ForUtil Approach*

This approach tends to make file format friendly to block decoding and decode 
block based on the efficient ForUtil (with SIMD opt) for each get. As a result 
we can get a rather delicious (130%) speed up in {{Browse*}} tasks. But we also 
get a slight (10%) regression in tasks that reading facets with a query (like 
MedTermDayTaxoFacets) since we are reading sparse values there and we need to 
decompress the whole 128 values block even we only need one value in that block.

Here is the [code|https://github.com/apache/lucene/pull/562] and luceneutil 
benchmark:
{code:java}
                            TaskQPS baseline      StdDevQPS my_modified_version 
     StdDev                Pct diff p-value
         AndHighMedDayTaxoFacets       71.49      (2.1%)       64.72      
(2.0%)   -9.5% ( -13% -   -5%) 0.000
            MedTermDayTaxoFacets       25.79      (2.6%)       24.00      
(1.8%)   -6.9% ( -11% -   -2%) 0.000
        AndHighHighDayTaxoFacets       13.13      (3.4%)       12.63      
(3.1%)   -3.9% ( -10% -    2%) 0.000
          OrHighMedDayTaxoFacets       13.71      (4.1%)       13.41      
(4.7%)   -2.2% ( -10% -    6%) 0.118
                        PKLookup      204.87      (3.9%)      203.03      
(3.6%)   -0.9% (  -8% -    6%) 0.450
                         Prefix3      113.85      (3.6%)      113.32      
(4.6%)   -0.5% (  -8% -    8%) 0.724
                    HighSpanNear       25.34      (2.5%)       25.26      
(3.1%)   -0.3% (  -5% -    5%) 0.714
                     LowSpanNear       55.96      (2.0%)       55.80      
(2.1%)   -0.3% (  -4% -    3%) 0.658
                     MedSpanNear       56.84      (2.4%)       56.90      
(2.2%)    0.1% (  -4% -    4%) 0.895
                 MedSloppyPhrase       26.57      (1.8%)       26.60      
(1.9%)    0.1% (  -3% -    3%) 0.831
                HighSloppyPhrase       30.20      (3.7%)       30.24      
(3.6%)    0.2% (  -6% -    7%) 0.890
                       OrHighMed       49.96      (2.1%)       50.06      
(1.7%)    0.2% (  -3% -    4%) 0.742
                      AndHighMed       96.70      (2.9%)       96.95      
(2.6%)    0.3% (  -5% -    5%) 0.772
             LowIntervalsOrdered       23.32      (4.6%)       23.38      
(4.5%)    0.3% (  -8% -    9%) 0.856
                      OrHighHigh       38.09      (1.9%)       38.20      
(1.8%)    0.3% (  -3% -    4%) 0.643
                      TermDTSort      128.55     (14.7%)      128.94     
(11.6%)    0.3% ( -22% -   31%) 0.942
                          Fuzzy1       99.54      (7.1%)       99.86      
(8.0%)    0.3% ( -13% -   16%) 0.893
            HighIntervalsOrdered       15.58      (2.6%)       15.65      
(2.6%)    0.4% (  -4% -    5%) 0.636
                         Respell       63.96      (1.9%)       64.22      
(2.3%)    0.4% (  -3% -    4%) 0.542
                   OrHighNotHigh      611.12      (5.8%)      613.85      
(6.2%)    0.4% ( -10% -   13%) 0.814
             MedIntervalsOrdered       59.48      (5.2%)       59.75      
(5.1%)    0.5% (  -9% -   11%) 0.780
                     AndHighHigh       58.76      (3.0%)       59.16      
(3.0%)    0.7% (  -5% -    6%) 0.478
                   OrNotHighHigh      619.53      (6.0%)      623.79      
(7.1%)    0.7% ( -11% -   14%) 0.740
                      HighPhrase       31.00      (2.5%)       31.26      
(2.7%)    0.8% (  -4% -    6%) 0.307
                      AndHighLow      828.41      (5.9%)      835.65      
(7.1%)    0.9% ( -11% -   14%) 0.672
                    OrNotHighLow      986.46      (6.8%)      995.13     
(10.5%)    0.9% ( -15% -   19%) 0.752
            HighTermTitleBDVSort      110.39     (12.3%)      111.38     
(11.1%)    0.9% ( -20% -   27%) 0.807
                          IntNRQ      151.29      (2.6%)      152.96      
(3.5%)    1.1% (  -4% -    7%) 0.262
                         LowTerm     1876.18      (7.8%)     1897.19      
(8.3%)    1.1% ( -13% -   18%) 0.660
           HighTermDayOfYearSort      108.34     (18.9%)      109.87     
(17.4%)    1.4% ( -29% -   46%) 0.805
               HighTermMonthSort       65.84     (11.0%)       66.78     
(11.7%)    1.4% ( -19% -   27%) 0.689
                    OrHighNotMed      770.05      (5.3%)      782.54      
(8.8%)    1.6% ( -11% -   16%) 0.480
                        Wildcard      182.10      (5.5%)      185.24      
(7.2%)    1.7% ( -10% -   15%) 0.394
                 LowSloppyPhrase       33.75      (6.6%)       34.35      
(8.8%)    1.8% ( -12% -   18%) 0.478
                       MedPhrase      161.57      (3.8%)      164.62      
(6.1%)    1.9% (  -7% -   12%) 0.242
                    OrHighNotLow      679.46      (7.2%)      693.59      
(7.6%)    2.1% ( -11% -   18%) 0.374
                    OrNotHighMed      690.91      (7.4%)      706.15      
(8.8%)    2.2% ( -13% -   19%) 0.390
                        HighTerm     1388.14      (6.3%)     1420.26      
(7.8%)    2.3% ( -11% -   17%) 0.302
                       LowPhrase      410.16      (5.0%)      420.38      
(5.0%)    2.5% (  -7% -   13%) 0.114
                       OrHighLow      479.96      (5.1%)      492.39      
(5.7%)    2.6% (  -7% -   14%) 0.128
                         MedTerm     1575.41      (5.9%)     1618.88      
(8.2%)    2.8% ( -10% -   17%) 0.221
                          Fuzzy2       64.75      (8.3%)       66.76      
(8.3%)    3.1% ( -12% -   21%) 0.237
           BrowseMonthTaxoFacets       14.39     (12.1%)       18.58     
(17.3%)   29.1% (   0% -   66%) 0.000
     BrowseRandomLabelTaxoFacets       12.01      (8.5%)       17.01     
(18.2%)   41.6% (  13% -   74%) 0.000
            BrowseDateTaxoFacets       13.72     (11.2%)       19.83     
(26.5%)   44.5% (   6% -   92%) 0.000
       BrowseDayOfYearTaxoFacets       13.84     (11.5%)       20.03     
(27.4%)   44.8% (   5% -   94%) 0.000
     BrowseRandomLabelSSDVFacets       10.31      (2.6%)       17.72      
(4.2%)   71.9% (  63% -   80%) 0.000
           BrowseMonthSSDVFacets       15.56      (3.3%)       34.58     
(12.3%)  122.3% ( 103% -  142%) 0.000
       BrowseDayOfYearSSDVFacets       14.17      (2.9%)       32.91     
(11.6%)  132.3% ( 114% -  151%) 0.000
{code}
*Detect Warm Approach*

This approach is keeping the origin file format to keep the high efficiency of 
reading sparse values, but try to detect a dense reading and do some block 
decoding for these situations. To be specific, we assume user is reading dense 
values if more than 80% values were required in the first block. And then we 
can use block decoding for following gets. The approach is limited to be used 
in forward reading cases and won't be as efficient as ForUtil in block decoding 
but it is nearly a net win.

Here is the [code|https://github.com/apache/lucene/pull/570] and luceneutil 
benchmark:
{code:java}
                            TaskQPS baseline      StdDevQPS my_modified_version 
     StdDev                Pct diff p-value
            MedTermDayTaxoFacets       55.17      (6.5%)       53.38      
(7.1%)   -3.2% ( -15% -   11%) 0.129
                        Wildcard      309.31     (12.9%)      299.80     
(12.6%)   -3.1% ( -25% -   25%) 0.446
                    OrNotHighLow      696.18      (8.7%)      677.46     
(10.1%)   -2.7% ( -19% -   17%) 0.367
                        HighTerm     1183.05      (9.5%)     1151.67      
(9.2%)   -2.7% ( -19% -   17%) 0.368
                       OrHighMed      120.60      (7.0%)      117.55      
(7.8%)   -2.5% ( -16% -   13%) 0.279
          OrHighMedDayTaxoFacets        9.46      (7.5%)        9.25      
(6.7%)   -2.2% ( -15% -   12%) 0.320
                         Prefix3      177.41      (7.3%)      173.96     
(10.3%)   -1.9% ( -18% -   16%) 0.489
        AndHighHighDayTaxoFacets       28.81      (6.7%)       28.35      
(6.3%)   -1.6% ( -13% -   12%) 0.433
           BrowseMonthTaxoFacets       13.50     (13.4%)       13.30      
(6.6%)   -1.5% ( -18% -   21%) 0.658
         AndHighMedDayTaxoFacets       46.43      (7.8%)       45.75      
(7.3%)   -1.5% ( -15% -   14%) 0.540
                       MedPhrase      360.70      (8.5%)      355.47      
(8.3%)   -1.4% ( -16% -   16%) 0.587
                      AndHighMed      233.52      (6.7%)      230.19      
(7.0%)   -1.4% ( -14% -   13%) 0.510
            HighTermTitleBDVSort       72.17     (16.9%)       71.14     
(15.4%)   -1.4% ( -28% -   37%) 0.780
                    OrHighNotMed      659.68      (9.7%)      650.38     
(12.6%)   -1.4% ( -21% -   23%) 0.691
                      HighPhrase       73.05      (7.6%)       72.25      
(9.3%)   -1.1% ( -16% -   17%) 0.685
                      TermDTSort      123.29     (15.5%)      122.10     
(13.8%)   -1.0% ( -26% -   33%) 0.835
                          IntNRQ      167.75      (7.4%)      166.17      
(8.6%)   -0.9% ( -15% -   16%) 0.710
                   OrHighNotHigh      890.84     (13.2%)      883.31     
(11.5%)   -0.8% ( -22% -   27%) 0.828
                       OrHighLow      279.24      (7.5%)      276.97      
(6.6%)   -0.8% ( -13% -   14%) 0.718
                        PKLookup      198.13      (6.6%)      196.54      
(6.9%)   -0.8% ( -13% -   13%) 0.707
                 MedSloppyPhrase       94.28      (8.0%)       93.55      
(6.5%)   -0.8% ( -14% -   14%) 0.737
                      AndHighLow      574.70      (8.2%)      570.50      
(9.6%)   -0.7% ( -17% -   18%) 0.795
                    OrNotHighMed      717.34     (11.5%)      712.33     
(12.4%)   -0.7% ( -22% -   26%) 0.853
                     AndHighHigh       61.26      (7.2%)       60.84      
(6.3%)   -0.7% ( -13% -   13%) 0.753
                HighSloppyPhrase        6.56      (6.5%)        6.52      
(5.3%)   -0.7% ( -11% -   11%) 0.729
                 LowSloppyPhrase      159.12      (6.9%)      158.23      
(6.5%)   -0.6% ( -13% -   13%) 0.794
                       LowPhrase       88.55      (8.6%)       88.07      
(8.6%)   -0.5% ( -16% -   18%) 0.844
                     MedSpanNear       14.63      (6.1%)       14.55      
(5.3%)   -0.5% ( -11% -   11%) 0.786
                          Fuzzy2       24.31      (9.5%)       24.19      
(7.7%)   -0.5% ( -16% -   18%) 0.858
                         MedTerm     1440.59      (9.8%)     1433.53     
(11.5%)   -0.5% ( -19% -   23%) 0.885
                    HighSpanNear       23.52      (6.1%)       23.40      
(5.9%)   -0.5% ( -11% -   12%) 0.797
                      OrHighHigh       32.01      (8.4%)       31.96      
(5.7%)   -0.1% ( -13% -   15%) 0.948
                          Fuzzy1       82.31     (11.7%)       82.30     
(13.5%)   -0.0% ( -22% -   28%) 0.998
                    OrHighNotLow      724.27      (9.5%)      724.70     
(10.1%)    0.1% ( -17% -   21%) 0.985
           HighTermDayOfYearSort      156.00     (14.4%)      156.22     
(14.6%)    0.1% ( -25% -   33%) 0.975
                         Respell       68.40      (8.2%)       68.49      
(7.6%)    0.1% ( -14% -   17%) 0.955
             MedIntervalsOrdered        9.22      (7.4%)        9.23      
(6.9%)    0.2% ( -13% -   15%) 0.936
                   OrNotHighHigh      571.66      (8.1%)      572.72     
(10.9%)    0.2% ( -17% -   20%) 0.951
                     LowSpanNear       82.39      (7.1%)       82.57      
(4.4%)    0.2% ( -10% -   12%) 0.907
                         LowTerm     1355.15     (10.2%)     1358.89     
(10.0%)    0.3% ( -18% -   22%) 0.931
               HighTermMonthSort       58.72     (20.8%)       58.95     
(20.1%)    0.4% ( -33% -   52%) 0.950
             LowIntervalsOrdered      117.01      (7.2%)      117.50      
(6.2%)    0.4% ( -12% -   14%) 0.844
            HighIntervalsOrdered       13.76      (8.8%)       13.82      
(6.8%)    0.5% ( -13% -   17%) 0.841
     BrowseRandomLabelTaxoFacets       11.40     (10.0%)       13.34     
(15.0%)   17.0% (  -7% -   46%) 0.000
       BrowseDayOfYearTaxoFacets       13.32     (11.5%)       15.71     
(15.0%)   17.9% (  -7% -   50%) 0.000
            BrowseDateTaxoFacets       13.15     (11.3%)       15.64     
(14.6%)   18.9% (  -6% -   50%) 0.000
     BrowseRandomLabelSSDVFacets       10.03      (7.9%)       12.20     
(10.1%)   21.6% (   3% -   43%) 0.000
           BrowseMonthSSDVFacets       15.27      (6.9%)       19.16     
(10.8%)   25.4% (   7% -   46%) 0.000
       BrowseDayOfYearSSDVFacets       13.88      (6.0%)       18.71     
(11.8%)   34.7% (  15% -   55%) 0.000
{code}
I wonder which way you will prefer? or if you want to try some other ideas but 
having no time I'm willing to help too. Any feedback will be welcome :)

> Introduce a BlockReader based on ForUtil and use it for NumericDocValues
> ------------------------------------------------------------------------
>
>                 Key: LUCENE-10334
>                 URL: https://issues.apache.org/jira/browse/LUCENE-10334
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/codecs
>            Reporter: Feng Guo
>            Priority: Major
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> Previous talk is here: [https://github.com/apache/lucene/pull/557]
> This is trying to add a new BlockReader based on ForUtil to replace the 
> DirectReader we are using for NumericDocvalues
> -*Benchmark based on wiki10m*- (Previous benchmark results are wrong so i 
> deleted it to avoid misleading, let's see the benchmark in comments.)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to