lszskye opened a new pull request, #573: URL: https://github.com/apache/paimon-cpp/pull/573
### Purpose Linked issue: #452 Scalar index evaluation previously rechecked indexed rows without distinguishing exact results from potentially incomplete results. Root search limits were also applied per file, so reading multiple files could return more than the requested TopK. This change: - Adds result-level exactness to FileIndexResult and defines its propagation through And/Or. - Uses the first exact scalar index result. If none is available, scans the original candidate scope to recover matches that an inexact index may omit. - Passes eligible rows between index queries and pushes scalar predicates into fallback data reads. - Adds TopKBatchReader to select and order root search results across all files and supplied splits. Internal score columns are removed when not projected. - Normalizes dictionary-encoded columns before combining candidates, allowing files with different physical string encodings to participate in the same TopK. - Global TopK with fallback branches is rejected. - Root vector searches require an explicit distance type. ### Tests Coverage includes: - Result exactness and And/Or behavior. - First-exact index selection and candidate propagation. - Timestamp precision loss, recovery of missing rows, and avoidance of unnecessary scalar rechecks. - Global TopK across files and splits, score projection, metric ordering, ties, batching, and deletion vectors. ### API and Format - Adds FileIndexResult::IsExact() and makes And/Or pure virtual. Custom subclasses must initialize exactness and implement both methods. - Extends bitmap/scored result construction with exactness. - Adds FileIndexReader::Evaluate(predicate, candidates). - Requires distance_type for root vector searches. - Defines root search limits and ordering across all splits supplied to TableRead::CreateReader. ### Documentation Updates the file-index guide and public API comments for result exactness. ### Generative AI tooling Generated-by: OpenAI Codex -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
