ercsonusharma commented on PR #4645:
URL: https://github.com/apache/solr/pull/4645#issuecomment-5047110029
Thanks for the review, @gerlowskija
Sure, here are the answers based on what I found:
**1. Binary vs. SortedSet DocValues** - Yes, I implemented the SortedSet
version first as it's the more common Solr pattern and natively multiValued but
later, switched to Binary after benchmarking. With SortedSet the docValues path
regressed as the leading clause broadened. Once the lead matched enough docs to
verify and well below the rangeCost/8 cutover, the IndexOrDocValuesQuery chose
docValues. The query ran ~2× slower than plain BKD which might grow as we reach
towards the cutover threshold. SortedSet builds a large term dictionary and
every per-doc verify pays a lookupOrd to recover the packed bytes, costlier
than the sequential BKD scan the docValues path is meant to replace.
BinaryDocValues stores the packed [min…|max…] blob directly on the document,
read with no dictionary/ordinal indirection, which flips the selective case
from a regression to a clear win.
SortedSet backed benchmark, selective lead ~5k docs (below the 62.5k
cutover), the docValues path ran ~2× slower than plain BKD:
```
Benchmark (criteria) (field) (scenario)
Mode Cnt Score Error Units
RangeSearch.rangeQuery contains r_ir selective_dv thrpt 8
1542.278 ± 146.843 ops/s
RangeSearch.rangeQuery contains r_ir_dv selective_dv thrpt 8
728.939 ± 61.275 ops/s
```
**2. Sorting and Faceting** - Combination of both actually,
- IMO, Sorting is a genuine semantic limitation, not scope. A range value is
a multi-dimensional box with no single natural sort key - sort by min(?) by
max(?) by width(?) That said, it has potential as a follow-up feature to take
up later.
- Faceting is more a scope call. The docValues we add are opaque packed
blobs for the filter path only - no scalar/ordinal value, so facet.field can't
consume them. Where the distinct ranges are known, facet.query already covers
bucketing and each is a range query, so it rides the same optimization.
Overall, the docValues here are scoped deliberately to the
IndexOrDocValuesQuery filter optimization: sort is semantically undefined for a
range (could be taken up as a follow up), and field faceting would need a
different DV shape than the packed blob.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
