ercsonusharma opened a new pull request, #4645: URL: https://github.com/apache/solr/pull/4645
https://issues.apache.org/jira/browse/SOLR-18201 # Description The numeric range field types i.e. IntRangeField, LongRangeField, FloatRangeField, and DoubleRangeField, did not support docValues. This PR adds opt-in, single/multiValued docValues to all four. The motivation is a query-time optimization. Today a range clause (e.g. my_range:[100 TO 200]) always walks the field's Points (BKD) tree and materializes a bitset of all matching docs, regardless of how selective the rest of the query is. When such a clause is AND-ed with a much more selective clause (myStrField:someSelectiveValue AND my_range:[100 TO 200]), this is wasteful, we materialize a large bitset only to intersect it with a handful of candidates. # Solution With docValues present, the range clause is wrapped in Lucene's IndexOrDocValuesQuery, which lets the selective clause lead iteration and verifies the range predicate per-candidate from docValues, skipping the BKD materialization entirely. Also, introducec MultiBinaryRangeDocValuesQuery, backed by BinaryDocValues as SortedSet backing builds a huge term dictionary and every per-doc verify pays a lookupOrd making it more costly than the sequential BKD materialization it was meant to replace. It comes with minor modification in lucene document creation by packing multiValued field's ranges into a single blob. These field types are new/unreleased, so this is added without back-compat concerns. # Tests - Added new integration tests - NumericRangeDocValuesTest and NumericRangeMultiValuedDocValuesTest. - Existing numericrange tests are unaffected - Added a benchmark RangeSearch whose run results are as below: ``` Benchmark (criteria) (field) (scenario) Mode Cnt Score Error Units RangeSearch.rangeQuery contains r_ir selective_dv thrpt 5 1637.126 ± 33.185 ops/s RangeSearch.rangeQuery contains r_ir broad_points thrpt 5 318.608 ± 5.378 ops/s RangeSearch.rangeQuery contains r_ir none_points thrpt 5 433.680 ± 5.315 ops/s RangeSearch.rangeQuery contains r_ir_dv selective_dv thrpt 5 4953.934 ± 283.397 ops/s RangeSearch.rangeQuery contains r_ir_dv broad_points thrpt 5 314.803 ± 16.153 ops/s RangeSearch.rangeQuery contains r_ir_dv none_points thrpt 5 425.166 ± 16.021 ops/s RangeSearch.rangeQuery contains r_mv_ir selective_dv thrpt 5 934.336 ± 11.399 ops/s RangeSearch.rangeQuery contains r_mv_ir broad_points thrpt 5 315.280 ± 19.342 ops/s RangeSearch.rangeQuery contains r_mv_ir none_points thrpt 5 267.174 ± 8.491 ops/s RangeSearch.rangeQuery contains r_mv_ir_dv selective_dv thrpt 5 2714.818 ± 40.461 ops/s RangeSearch.rangeQuery contains r_mv_ir_dv broad_points thrpt 5 322.387 ± 2.730 ops/s RangeSearch.rangeQuery contains r_mv_ir_dv none_points thrpt 5 268.392 ± 4.039 ops/s RangeSearch.rangeQuery intersects r_ir selective_dv thrpt 5 1899.224 ± 60.904 ops/s RangeSearch.rangeQuery intersects r_ir broad_points thrpt 5 324.266 ± 9.377 ops/s RangeSearch.rangeQuery intersects r_ir none_points thrpt 5 438.226 ± 59.882 ops/s RangeSearch.rangeQuery intersects r_ir_dv selective_dv thrpt 5 4073.935 ± 903.913 ops/s RangeSearch.rangeQuery intersects r_ir_dv broad_points thrpt 5 313.721 ± 17.786 ops/s RangeSearch.rangeQuery intersects r_ir_dv none_points thrpt 5 441.639 ± 20.801 ops/s RangeSearch.rangeQuery intersects r_mv_ir selective_dv thrpt 5 1038.356 ± 25.230 ops/s RangeSearch.rangeQuery intersects r_mv_ir broad_points thrpt 5 326.816 ± 21.920 ops/s RangeSearch.rangeQuery intersects r_mv_ir none_points thrpt 5 267.894 ± 15.156 ops/s RangeSearch.rangeQuery intersects r_mv_ir_dv selective_dv thrpt 5 2676.503 ± 203.337 ops/s RangeSearch.rangeQuery intersects r_mv_ir_dv broad_points thrpt 5 329.054 ± 5.202 ops/s RangeSearch.rangeQuery intersects r_mv_ir_dv none_points thrpt 5 269.826 ± 17.403 ops/s RangeSearch.rangeQuery within r_ir selective_dv thrpt 5 1937.916 ± 195.623 ops/s RangeSearch.rangeQuery within r_ir broad_points thrpt 5 306.693 ± 12.386 ops/s RangeSearch.rangeQuery within r_ir none_points thrpt 5 449.107 ± 24.540 ops/s RangeSearch.rangeQuery within r_ir_dv selective_dv thrpt 5 4862.166 ± 678.356 ops/s RangeSearch.rangeQuery within r_ir_dv broad_points thrpt 5 311.665 ± 30.428 ops/s RangeSearch.rangeQuery within r_ir_dv none_points thrpt 5 449.009 ± 10.885 ops/s RangeSearch.rangeQuery within r_mv_ir selective_dv thrpt 5 1089.617 ± 45.707 ops/s RangeSearch.rangeQuery within r_mv_ir broad_points thrpt 5 332.549 ± 20.213 ops/s RangeSearch.rangeQuery within r_mv_ir none_points thrpt 5 279.219 ± 5.414 ops/s RangeSearch.rangeQuery within r_mv_ir_dv selective_dv thrpt 5 2518.521 ± 344.224 ops/s RangeSearch.rangeQuery within r_mv_ir_dv broad_points thrpt 5 336.337 ± 15.615 ops/s RangeSearch.rangeQuery within r_mv_ir_dv none_points thrpt 5 270.378 ± 10.551 ops/s RangeSearch.rangeQuery crosses r_ir selective_dv thrpt 5 1683.941 ± 398.317 ops/s RangeSearch.rangeQuery crosses r_ir broad_points thrpt 5 289.636 ± 18.353 ops/s RangeSearch.rangeQuery crosses r_ir none_points thrpt 5 440.876 ± 11.024 ops/s RangeSearch.rangeQuery crosses r_ir_dv selective_dv thrpt 5 4366.407 ± 1623.692 ops/s RangeSearch.rangeQuery crosses r_ir_dv broad_points thrpt 5 302.358 ± 38.972 ops/s RangeSearch.rangeQuery crosses r_ir_dv none_points thrpt 5 452.579 ± 9.217 ops/s RangeSearch.rangeQuery crosses r_mv_ir selective_dv thrpt 5 1031.718 ± 36.046 ops/s RangeSearch.rangeQuery crosses r_mv_ir broad_points thrpt 5 319.433 ± 15.376 ops/s RangeSearch.rangeQuery crosses r_mv_ir none_points thrpt 5 276.894 ± 16.116 ops/s RangeSearch.rangeQuery crosses r_mv_ir_dv selective_dv thrpt 5 2452.196 ± 319.052 ops/s RangeSearch.rangeQuery crosses r_mv_ir_dv broad_points thrpt 5 330.320 ± 18.011 ops/s RangeSearch.rangeQuery crosses r_mv_ir_dv none_points thrpt 5 276.039 ± 17.509 ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
