CrownChu commented on code in PR #10771:
URL: https://github.com/apache/paimon/pull/10771#discussion_r4379386371
##########
paimon-core/src/main/java/org/apache/paimon/globalindex/GlobalIndexQuery.java:
##########
@@ -330,8 +330,9 @@ private static GlobalIndexQuery createForIndexedField(
}
List<IndexGroup> selectedGroups = new ArrayList<>();
for (IndexGroup group : fieldGroups) {
- // Scalar lookups use independent single-field index definitions.
- if (group.extraFields.isEmpty()) {
+ // Composite-key indexes require their own planner, not
independent field lookups.
+ if (group.extraFields.isEmpty()
+ ||
GlobalIndexerFactoryUtils.supportsGroupedIndexScalarQueries(group.type)) {
Review Comment:
Thanks, confirmed and fixed in
[8e12fe3cb6](https://github.com/apache/paimon/commit/8e12fe3cb65d6c310260ee8ef6b38ad59192210f).
Unsupported ES scalar operations now return `Optional.empty()` instead of
`conservativeAllRows()`, including the guarded capability paths and runtime
unsupported-operation fallbacks. A supported query with zero hits still returns
an empty bitmap inside a present Optional. Negation propagates unsupported
rather than complementing an unlabeled superset. Such results no longer claim
evaluated coverage or exactness.
The real-archive regressions cover ordinary scalar reads, coverage, AND/OR,
and vector/full-text prefilters. With the FULL/refine-enabled fixture described
here, the scalar oracle is `{2,8,11}`, vector top-1 is `2`, and full-text
`keep` returns `{2,8}`. Same-field partial conjunctions are covered as well.
Exercising the unsupported-filter fallback also exposed a metric issue: raw
vector scoring could use current table options instead of the persisted index
metric. The fallback now retains the persisted metric while preserving each
backend's existing shard-conflict policy; cosine-vs-L2 and backend-policy
regressions are included.
This is a correctness-only fix: ESLib stays at 1.0.7, and keyword-range
capability expansion is deferred to a separate change. The targeted clean
reactor passed **458 tests, 0 failures/errors/skips**, with normal Maven checks
enabled. This is not a claim that full GitHub CI or cloud E2E has passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
