deepthi912 opened a new pull request, #19153:
URL: https://github.com/apache/pinot/pull/19153
## Summary
When a filter predicate targets a star-tree dimension whose forward index is
`RAW`-encoded but a separate dictionary is present, star-tree eligibility
passes and execution then crashes with:
```
Caused by: java.lang.UnsupportedOperationException
at
BaseRawValueBasedPredicateEvaluator.getMatchingDictIds(BaseRawValueBasedPredicateEvaluator.java:48)
at
StarTreeFilterOperator.getMatchingDictIds(StarTreeFilterOperator.java:458)
at StarTreeFilterOperator.traverseStarTree(StarTreeFilterOperator.java:286)
```
## Root cause
`PredicateEvaluatorProvider.getDictionaryUsableForFiltering` discards the
dictionary when the forward index is RAW and no dict-consuming scan operator
(inverted / exact range index) is available for the predicate type. It has no
awareness that star-tree is *also* a dict-consuming filter operator.
Consequently a raw-value evaluator is built for star-tree dimensions with RAW
forward + separated dictionary. Star-tree traversal invokes
`getMatchingDictIds()` / `applySV(int)` on the evaluator — both throw
`UnsupportedOperationException` on `BaseRawValueBasedPredicateEvaluator`.
## Fix
`StarTreeUtils.getPredicateEvaluator` already verifies that the underlying
`DataSource` has a dictionary (line 320-323). Extend the lookup path so that
when the pre-built evaluator is raw-value-based, it is rebuilt as
dictionary-based against the segment dictionary using the second overload of
`PredicateEvaluatorProvider.getPredicateEvaluator(Predicate, Dictionary,
DataType, QueryContext)`, which accepts the dictionary directly and bypasses
the "no dict-consuming operator, discard dict" gate.
The upgrade is a no-op for already dict-based evaluators (checked via
`instanceof BaseRawValueBasedPredicateEvaluator`), so existing
DICTIONARY-forward tables see no change. The raw-value evaluator that was
originally built remains available for the scan-fallback path (which never
touches star-tree). Star-tree receives its own dict-based copy scoped to the
star-tree operator.
## Behavior after fix
| Column config | Before | After |
|---|---|---|
| DICTIONARY forward + dict | ✅ Star-tree works | ✅ Same (upgrade no-op) |
| RAW forward + separated dict | ❌ UnsupportedOperationException | ✅
Star-tree works |
| RAW forward + no dict | ❌ Rejected earlier at line 320-323 | ✅ Same
(unaffected) |
| Scan-fallback path (non-star-tree) | ✅ Raw eval works | ✅ Same
(unaffected) |
Applies to all star-tree-supported predicate types: `EQ`, `NOT_EQ`, `IN`,
`NOT_IN`, `RANGE`. Both SSE and MSE benefit since both share the same
leaf-stage star-tree code path.
## Performance parity
Star-tree query execution reads column data via
`_starTreeV2.getDataSource(column)` — the star-tree's own dict-encoded internal
forward indexes for every dim and aggregation column. The segment's main-column
forward index is never touched during star-tree execution. Consequently,
RAW-forward-with-dict star-tree queries now match DICTIONARY-forward star-tree
query performance for identical query shapes.
## Test plan
- [x] `./mvnw -pl pinot-core -Dtest="Star*Test,*StarTree*Test" test` — 66
tests pass
- [x] `./mvnw -pl pinot-core spotless:apply checkstyle:check` — clean
- [ ] Integration test to be added in a follow-up PR: build a segment with
`encodingType: RAW` and `indexes.dictionary.disabled: false` on a star-tree
dimension, verify EQ / IN / RANGE queries succeed with STAR_TREE plan node and
match DICTIONARY-encoded baseline results
## Draft note
Opening as draft to allow reviewer feedback on the approach (single-point
upgrade at STI-planner boundary vs alternatives like extending
`PredicateEvaluatorProvider.getDictionaryUsableForFiltering` to recognize STI
as a dict-consuming operator). The chosen approach keeps the change localized
to STI and avoids touching the scan-path planner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
