JingsongLi commented on code in PR #10771:
URL: https://github.com/apache/paimon/pull/10771#discussion_r4385576271
##########
paimon-core/src/main/java/org/apache/paimon/globalindex/DataEvolutionGlobalIndexScanner.java:
##########
@@ -231,7 +231,12 @@ public static Optional<DataEvolutionGlobalIndexScanner>
createForScalarFilters(
pinnedSnapshot,
partitionFilter,
indexFiles.stream()
- .filter(file -> !isMultiFieldIndex(file))
+ .filter(
+ file ->
+ !isMultiFieldIndex(file)
+ || GlobalIndexerFactoryUtils
Review Comment:
[P2] Keep vector readers separate for different grouped definitions
Admitting grouped scalar filters here makes a previously working filtered
vector query take the ANN path, which still creates one indexer from the first
split and reuses it for every definition. I reproduced this with the real
`ESIndexScalarTableTest` fixture: build rows `[0,6)` as `(embedding, category,
price)` and `[6,12)` as `(embedding, category)`, then search `vector(5)` with
`category = 'c1'` and limit 1. Instead of returning row 4, the second archive
fails with `Invalid es-index metadata`, caused by `ES index field count does
not match current schema: indexed=2, current=3`. Replacing only the ES factory
with its exact parent version restores row 4 for the same test, since the
grouped scalar path then uses raw fallback.
Please create/reuse the normal ANN indexer by index type and ordered field
definition, as the new fallback-metric path and existing full-text path already
do, or fall back when these definitions differ. Add this supported-equality
case alongside the new unsupported-filter test across different definitions;
that test currently exercises only the raw path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
