CrownChu commented on code in PR #10771:
URL: https://github.com/apache/paimon/pull/10771#discussion_r4409448800
##########
paimon-core/src/main/java/org/apache/paimon/globalindex/DataEvolutionGlobalIndexScanner.java:
##########
@@ -231,7 +231,12 @@ public static Optional<DataEvolutionGlobalIndexScanner>
createForScalarFilters(
pinnedSnapshot,
partitionFilter,
indexFiles.stream()
- .filter(file -> !isMultiFieldIndex(file))
+ .filter(
+ file ->
+ !isMultiFieldIndex(file)
+ || GlobalIndexerFactoryUtils
Review Comment:
Thanks, confirmed and fixed in
[59da228582](https://github.com/apache/paimon/commit/59da228582a16c1bebd0ca794b3bcacf8226e5db),
kept as a separate commit for review.
Both single and batch ANN reads now select indexers by index type and
ordered field IDs. Matching shards share an indexer; different definitions use
separate instances. Range-based splits can also contain multiple definitions,
so those are separated internally before creating readers, preserving the
original row ranges and offsets without changing scan-plan serialization.
After the readers have registered their metadata, the final metrics must
agree across definitions before merging scores, reranking or combining raw
rows. Same-definition shards retain their backend's existing metric-conflict
policy.
The real-archive regression now returns row 4 for the supported `category =
'c1'`, vector(5), limit-1 case. Added coverage includes single/batch queries,
no-filter and supported/unsupported filters, identical and overlapping ranges,
reordered fields, raw tails, persisted cosine with current L2 options, and
incompatible metrics. The final 20 new test cases reproduce 14 failures/errors
against the old production code and all pass with the fix.
Validation: **478 targeted tests passed, 0 failures/errors/skips**, using
JDK 11 and a normal-check `clean package` reactor build (common 56, core 274,
ESLib 142, Lumina 6; no `fast-build`). This is local targeted validation, not a
full GitHub CI or cloud E2E result. Please take another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
