[
https://issues.apache.org/jira/browse/LUCENE-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12664661#action_12664661
]
Michael McCandless commented on LUCENE-1483:
--------------------------------------------
I decided to change the wiki indexes I use for perf. testing. Previously
the 10 and 100 segment indexes all had fix-sized segments, which is
not natural -- normally segments have a logarithmic size distribution.
So I created new 10 and 36 segment indexes from Wikipedia, with 2M
docs that are more "natural", and re-ran tests on them.
I think net/net ORDDEM2 makes a good default policy!
queue=1000
||numSeg||index||sortBy||method||query||topN||hits||warm||qps||warmnew||qpsnew||pctg||
|1|simple|country|val|text|1000|2000000| 0.9| 10.0| 0.7| 12.0| 20.0%|
|1|simple|country|ordsub|text|1000|2000000| 0.9| 10.0| 0.6| 13.6| 36.0%|
|1|simple|country|ordval|text|1000|2000000| 0.9| 10.0| 0.6| 13.9| 39.0%|
|1|simple|country|orddem|text|1000|2000000| 0.9| 10.0| 0.7| 13.8| 38.0%|
|1|simple|country|orddem2|text|1000|2000000| 0.9| 10.0| 0.6| 13.9| 39.0%|
|1|wiki|title|val|147|1000| 4984| 2.1| 739.1| 2.3| 379.7|-48.6%|
|1|wiki|title|ordsub|147|1000| 4984| 2.1| 739.1| 2.1| 954.7| 29.2%|
|1|wiki|title|ordval|147|1000| 4984| 2.1| 739.1| 2.1| 951.3| 28.7%|
|1|wiki|title|orddem|147|1000| 4984| 2.1| 739.1| 2.1| 858.8| 16.2%|
|1|wiki|title|orddem2|147|1000| 4984| 2.1| 739.1| 2.1| 860.7| 16.5%|
|1|wiki|title|val|text|1000| 97191| 2.1| 109.5| 2.4| 33.1|-69.8%|
|1|wiki|title|ordsub|text|1000| 97191| 2.1| 109.5| 2.1| 125.8| 14.9%|
|1|wiki|title|ordval|text|1000| 97191| 2.1| 109.5| 2.1| 126.1| 15.2%|
|1|wiki|title|orddem|text|1000| 97191| 2.1| 109.5| 2.1| 123.4| 12.7%|
|1|wiki|title|orddem2|text|1000| 97191| 2.1| 109.5| 2.1| 124.5| 13.7%|
|1|wiki|title|val|1|1000| 386435| 2.1| 46.5| 2.4| 12.7|-72.7%|
|1|wiki|title|ordsub|1|1000| 386435| 2.1| 46.5| 2.1| 60.7| 30.5%|
|1|wiki|title|ordval|1|1000| 386435| 2.1| 46.5| 2.1| 61.3| 31.8%|
|1|wiki|title|orddem|1|1000| 386435| 2.1| 46.5| 2.1| 61.0| 31.2%|
|1|wiki|title|orddem2|1|1000| 386435| 2.1| 46.5| 2.1| 60.8| 30.8%|
||numSeg||index||sortBy||method||query||topN||hits||warm||qps||warmnew||qpsnew||pctg||
|10|simple|country|val|text|1000|2000000| 0.8| 9.7| 0.6| 11.7| 20.6%|
|10|simple|country|ordsub|text|1000|2000000| 0.8| 9.7| 0.6| 13.4| 38.1%|
|10|simple|country|ordval|text|1000|2000000| 0.8| 9.7| 0.6| 13.6| 40.2%|
|10|simple|country|orddem|text|1000|2000000| 0.8| 9.7| 0.6| 13.6| 40.2%|
|10|simple|country|orddem2|text|1000|2000000| 0.8| 9.7| 0.6| 13.5| 39.2%|
|10|wiki|title|val|147|1000| 4984| 12.6| 659.2| 2.3| 406.4|-38.3%|
|10|wiki|title|ordsub|147|1000| 4984| 12.6| 659.2| 2.7| 68.6|-89.6%|
|10|wiki|title|ordval|147|1000| 4984| 12.6| 659.2| 2.1| 86.4|-86.9%|
|10|wiki|title|orddem|147|1000| 4984| 12.6| 659.2| 2.1| 117.8|-82.1%|
|10|wiki|title|orddem2|147|1000| 4984| 12.6| 659.2| 2.1| 288.0|-56.3%|
|10|wiki|title|val|text|1000| 97191| 12.7| 101.2| 2.4| 33.9|-66.5%|
|10|wiki|title|ordsub|text|1000| 97191| 12.7| 101.2| 2.7| 68.8|-32.0%|
|10|wiki|title|ordval|text|1000| 97191| 12.7| 101.2| 2.2| 83.1|-17.9%|
|10|wiki|title|orddem|text|1000| 97191| 12.7| 101.2| 2.2| 87.4|-13.6%|
|10|wiki|title|orddem2|text|1000| 97191| 12.7| 101.2| 2.2| 100.8| -0.4%|
|10|wiki|title|val|1|1000| 386435| 12.7| 42.3| 2.5| 14.5|-65.7%|
|10|wiki|title|ordsub|1|1000| 386435| 12.7| 42.3| 2.7| 46.4| 9.7%|
|10|wiki|title|ordval|1|1000| 386435| 12.7| 42.3| 2.2| 53.0| 25.3%|
|10|wiki|title|orddem|1|1000| 386435| 12.7| 42.3| 2.2| 54.7| 29.3%|
|10|wiki|title|orddem2|1|1000| 386435| 12.7| 42.3| 2.2| 57.4| 35.7%|
||numSeg||index||sortBy||method||query||topN||hits||warm||qps||warmnew||qpsnew||pctg||
|36|wiki|title|val|147|1000| 4984| 42.1| 609.6| 2.3| 358.5|-41.2%|
|36|wiki|title|ordsub|147|1000| 4984| 42.1| 609.6| 2.7| 37.6|-93.8%|
|36|wiki|title|ordval|147|1000| 4984| 42.1| 609.6| 2.2| 58.2|-90.5%|
|36|wiki|title|orddem|147|1000| 4984| 42.1| 609.6| 2.1| 113.6|-81.4%|
|36|wiki|title|orddem2|147|1000| 4984| 42.1| 609.6| 2.1| 258.2|-57.6%|
|36|wiki|title|val|text|1000| 97191| 42.4| 98.5| 2.4| 34.1|-65.4%|
|36|wiki|title|ordsub|text|1000| 97191| 42.4| 98.5| 2.7| 39.1|-60.3%|
|36|wiki|title|ordval|text|1000| 97191| 42.4| 98.5| 2.2| 65.1|-33.9%|
|36|wiki|title|orddem|text|1000| 97191| 42.4| 98.5| 2.2| 85.9|-12.8%|
|36|wiki|title|orddem2|text|1000| 97191| 42.4| 98.5| 2.2| 94.9| -3.7%|
|36|wiki|title|val|1|1000| 386435| 42.0| 44.2| 2.4| 14.5|-67.2%|
|36|wiki|title|ordsub|1|1000| 386435| 42.0| 44.2| 2.7| 30.7|-30.5%|
|36|wiki|title|ordval|1|1000| 386435| 42.0| 44.2| 2.2| 47.6| 7.7%|
|36|wiki|title|orddem|1|1000| 386435| 42.0| 44.2| 2.3| 54.9| 24.2%|
|36|wiki|title|orddem2|1|1000| 386435| 42.0| 44.2| 2.2| 56.8| 28.5%|
queue=10
||numSeg||index||sortBy||method||query||topN||hits||warm||qps||warmnew||qpsnew||pctg||
|1|simple|country|val|text|10|2000000| 0.6| 10.6| 0.7| 12.4| 17.0%|
|1|simple|country|ordsub|text|10|2000000| 0.6| 10.6| 0.6| 14.5| 36.8%|
|1|simple|country|ordval|text|10|2000000| 0.6| 10.6| 0.6| 14.6| 37.7%|
|1|simple|country|orddem|text|10|2000000| 0.6| 10.6| 0.6| 14.6| 37.7%|
|1|simple|country|orddem2|text|10|2000000| 0.6| 10.6| 0.6| 14.6| 37.7%|
|1|wiki|title|val|147|10| 4984| 2.1|3727.4| 2.3|2546.8|-31.7%|
|1|wiki|title|ordsub|147|10| 4984| 2.1|3727.4| 2.0|4537.5| 21.7%|
|1|wiki|title|ordval|147|10| 4984| 2.1|3727.4| 2.0|4568.1| 22.6%|
|1|wiki|title|orddem|147|10| 4984| 2.1|3727.4| 2.1|4552.5| 22.1%|
|1|wiki|title|orddem2|147|10| 4984| 2.1|3727.4| 2.0|4537.5| 21.7%|
|1|wiki|title|val|text|10| 97191| 2.1| 143.4| 2.3| 39.2|-72.7%|
|1|wiki|title|ordsub|text|10| 97191| 2.1| 143.4| 2.1| 165.5| 15.4%|
|1|wiki|title|ordval|text|10| 97191| 2.1| 143.4| 2.1| 167.7| 16.9%|
|1|wiki|title|orddem|text|10| 97191| 2.1| 143.4| 2.1| 167.7| 16.9%|
|1|wiki|title|orddem2|text|10| 97191| 2.1| 143.4| 2.1| 168.2| 17.3%|
|1|wiki|title|val|1|10| 386435| 2.1| 51.8| 2.4| 13.7|-73.6%|
|1|wiki|title|ordsub|1|10| 386435| 2.1| 51.8| 2.1| 66.3| 28.0%|
|1|wiki|title|ordval|1|10| 386435| 2.1| 51.8| 2.1| 69.5| 34.2%|
|1|wiki|title|orddem|1|10| 386435| 2.1| 51.8| 2.1| 69.5| 34.2%|
|1|wiki|title|orddem2|1|10| 386435| 2.1| 51.8| 2.1| 69.5| 34.2%|
||numSeg||index||sortBy||method||query||topN||hits||warm||qps||warmnew||qpsnew||pctg||
|10|simple|country|val|text|10|2000000| 0.7| 10.2| 0.5| 12.2| 19.6%|
|10|simple|country|ordsub|text|10|2000000| 0.7| 10.2| 0.5| 14.2| 39.2%|
|10|simple|country|ordval|text|10|2000000| 0.7| 10.2| 0.5| 14.5| 42.2%|
|10|simple|country|orddem|text|10|2000000| 0.7| 10.2| 0.5| 14.4| 41.2%|
|10|simple|country|orddem2|text|10|2000000| 0.7| 10.2| 0.5| 14.4| 41.2%|
|10|wiki|title|val|147|10| 4984| 12.5|2943.0| 2.3|1761.1|-40.2%|
|10|wiki|title|ordsub|147|10| 4984| 12.5|2943.0| 2.1|3035.3| 3.1%|
|10|wiki|title|ordval|147|10| 4984| 12.5|2943.0| 2.1|3325.2| 13.0%|
|10|wiki|title|orddem|147|10| 4984| 12.5|2943.0| 2.1|3481.8| 18.3%|
|10|wiki|title|orddem2|147|10| 4984| 12.5|2943.0| 2.1|3574.1| 21.4%|
|10|wiki|title|val|text|10| 97191| 12.6| 139.2| 2.4| 39.3|-71.8%|
|10|wiki|title|ordsub|text|10| 97191| 12.6| 139.2| 2.1| 161.7| 16.2%|
|10|wiki|title|ordval|text|10| 97191| 12.6| 139.2| 2.2| 165.2| 18.7%|
|10|wiki|title|orddem|text|10| 97191| 12.6| 139.2| 2.1| 165.4| 18.8%|
|10|wiki|title|orddem2|text|10| 97191| 12.6| 139.2| 2.1| 166.3| 19.5%|
|10|wiki|title|val|1|10| 386435| 12.6| 50.4| 2.4| 15.5|-69.2%|
|10|wiki|title|ordsub|1|10| 386435| 12.6| 50.4| 2.1| 65.8| 30.6%|
|10|wiki|title|ordval|1|10| 386435| 12.6| 50.4| 2.1| 68.1| 35.1%|
|10|wiki|title|orddem|1|10| 386435| 12.6| 50.4| 2.1| 67.9| 34.7%|
|10|wiki|title|orddem2|1|10| 386435| 12.6| 50.4| 2.2| 67.8| 34.5%|
||numSeg||index||sortBy||method||query||topN||hits||warm||qps||warmnew||qpsnew||pctg||
|36|wiki|title|val|147|10| 4984| 42.0|2137.0| 2.3|1143.0|-46.5%|
|36|wiki|title|ordsub|147|10| 4984| 42.0|2137.0| 2.1|1641.5|-23.2%|
|36|wiki|title|ordval|147|10| 4984| 42.0|2137.0| 2.1|2046.0| -4.3%|
|36|wiki|title|orddem|147|10| 4984| 42.0|2137.0| 2.1|2203.9| 3.1%|
|36|wiki|title|orddem2|147|10| 4984| 42.0|2137.0| 2.1|2215.3| 3.7%|
|36|wiki|title|val|text|10| 97191| 42.2| 135.2| 2.4| 39.8|-70.6%|
|36|wiki|title|ordsub|text|10| 97191| 42.2| 135.2| 2.1| 149.9| 10.9%|
|36|wiki|title|ordval|text|10| 97191| 42.2| 135.2| 2.1| 154.9| 14.6%|
|36|wiki|title|orddem|text|10| 97191| 42.2| 135.2| 2.1| 154.9| 14.6%|
|36|wiki|title|orddem2|text|10| 97191| 42.2| 135.2| 2.2| 155.0| 14.6%|
|36|wiki|title|val|1|10| 386435| 41.9| 49.7| 2.4| 15.7|-68.4%|
|36|wiki|title|ordsub|1|10| 386435| 41.9| 49.7| 2.2| 65.9| 32.6%|
|36|wiki|title|ordval|1|10| 386435| 41.9| 49.7| 2.2| 67.4| 35.6%|
|36|wiki|title|orddem|1|10| 386435| 41.9| 49.7| 2.1| 67.9| 36.6%|
|36|wiki|title|orddem2|1|10| 386435| 41.9| 49.7| 2.2| 67.7| 36.2%|
> Change IndexSearcher multisegment searches to search each individual segment
> using a single HitCollector
> --------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1483
> URL: https://issues.apache.org/jira/browse/LUCENE-1483
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.9
> Reporter: Mark Miller
> Priority: Minor
> Attachments: LUCENE-1483-partial.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, sortBench.py, sortCollate.py
>
>
> FieldCache and Filters are forced down to a single segment reader, allowing
> for individual segment reloading on reopen.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
