[
https://issues.apache.org/jira/browse/LUCENE-9648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Guo updated LUCENE-9648:
-----------------------------
Description:
LUCENE-6539 introduced DocValuesNumbersQuery to benefit some special cases
(many terms/numbers and fewish matching hits). Inspired by LUCENE-7714, maybe
we can speed it up when meeting index sort since we can do a binary search to
bound the docvalues before iter it.
Here are some simple benchmarks:
*Based on 50,000,000 doc, each long in query can hit at least one doc:*
|LongsCount|PointInSetQuery(s/op)|IndexSortDocValuesNumbersQuery(s/op)|DocValuesNumbersQuery(s/op)|
|1000|0.144 ± 0.001|0.428 ± 0.008|1.159 ± 0.031|
|5000|0.404 ± 0.003|0.424 ± 0.008|1.343 ± 0.022|
|10000|0.661 ± 0.012|0.436 ± 0.008|1.372 ± 0.011|
|50000|1.319 ± 0.011|0.450 ± 0.005|1.173 ± 0.017|
|100000|1.328 ± 0.007|0.459 ± 0.003|1.160 ± 0.098|
*Based on 100,000,000 doc, each long in query can hit at least one doc:*
|LongsCount|PointInSetQuery(s/op)|IndexSortDocValuesNumbersQuery(s/op)|DocValuesNumbersQuery(s/op)|
|1000|0.189 ± 0.003|0.860 ± 0.028|2.408 ± 0.087|
|5000|0.715 ± 0.010|0.879 ± 0.010|2.642 ± 0.105|
|10000|1.075 ± 0.034|0.859 ± 0.010|2.629 ± 0.077|
|50000|2.652 ± 0.114|0.865 ± 0.009|2.737 ± 0.099|
|100000|2.905 ± 0.027|0.881 ± 0.005|2.820 ± 0.474|
was:
LUCENE-6539 introduced DocValuesNumbersQuery to benefit some special cases
(many terms/numbers and fewish matching hits). Inspired by LUCENE-7714, maybe
we can speed it up when meeting index sort since we can do a binary search to
bound the docvalues before iter it.
Here are some simple benchmarks:
*Based on 50,000,000 doc, each long in query can hit at least one doc:*
|LongsCount|PointInSetQuery(s/op)|IndexSortDocValuesNumbersQuery(s/op)|DocValuesNumbersQuery(s/op)|
|1000|0.144 ± 0.001|0.428 ± 0.008|1.159 ± 0.031|
|5000|0.404 ± 0.003|0.424 ± 0.008|1.343 ± 0.022 s/op|
|10000|0.661 ± 0.012|0.436 ± 0.008|1.372 ± 0.011|
|50000|1.319 ± 0.011|0.450 ± 0.005|1.173 ± 0.017|
|100000|1.328 ± 0.007|0.459 ± 0.003|1.160 ± 0.098|
*Based on 100,000,000 doc, each long in query can hit at least one doc:*
|LongsCount|PointInSetQuery(s/op)|IndexSortDocValuesNumbersQuery(s/op)|DocValuesNumbersQuery(s/op)|
|1000|0.189 ± 0.003|0.860 ± 0.028|2.408 ± 0.087|
|5000|0.715 ± 0.010|0.879 ± 0.010|2.642 ± 0.105|
|10000|1.075 ± 0.034|0.859 ± 0.010|2.629 ± 0.077|
|50000|2.652 ± 0.114|0.865 ± 0.009|2.737 ± 0.099|
|100000|2.905 ± 0.027|0.881 ± 0.005|2.820 ± 0.474|
> Add a numbers query in sandbox that takes advantage of index sorting.
> ---------------------------------------------------------------------
>
> Key: LUCENE-9648
> URL: https://issues.apache.org/jira/browse/LUCENE-9648
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/sandbox
> Reporter: Feng Guo
> Priority: Minor
>
> LUCENE-6539 introduced DocValuesNumbersQuery to benefit some special cases
> (many terms/numbers and fewish matching hits). Inspired by LUCENE-7714, maybe
> we can speed it up when meeting index sort since we can do a binary search to
> bound the docvalues before iter it.
> Here are some simple benchmarks:
> *Based on 50,000,000 doc, each long in query can hit at least one doc:*
> |LongsCount|PointInSetQuery(s/op)|IndexSortDocValuesNumbersQuery(s/op)|DocValuesNumbersQuery(s/op)|
> |1000|0.144 ± 0.001|0.428 ± 0.008|1.159 ± 0.031|
> |5000|0.404 ± 0.003|0.424 ± 0.008|1.343 ± 0.022|
> |10000|0.661 ± 0.012|0.436 ± 0.008|1.372 ± 0.011|
> |50000|1.319 ± 0.011|0.450 ± 0.005|1.173 ± 0.017|
> |100000|1.328 ± 0.007|0.459 ± 0.003|1.160 ± 0.098|
> *Based on 100,000,000 doc, each long in query can hit at least one doc:*
> |LongsCount|PointInSetQuery(s/op)|IndexSortDocValuesNumbersQuery(s/op)|DocValuesNumbersQuery(s/op)|
> |1000|0.189 ± 0.003|0.860 ± 0.028|2.408 ± 0.087|
> |5000|0.715 ± 0.010|0.879 ± 0.010|2.642 ± 0.105|
> |10000|1.075 ± 0.034|0.859 ± 0.010|2.629 ± 0.077|
> |50000|2.652 ± 0.114|0.865 ± 0.009|2.737 ± 0.099|
> |100000|2.905 ± 0.027|0.881 ± 0.005|2.820 ± 0.474|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]