lszskye opened a new pull request, #573:
URL: https://github.com/apache/paimon-cpp/pull/573

   ### Purpose
   Linked issue: #452 
   
   Scalar index evaluation previously rechecked indexed rows without 
distinguishing exact results from potentially incomplete results. Root search 
limits were also applied per file, so reading multiple files could return more 
than the requested TopK.
   
   This change:
   - Adds result-level exactness to FileIndexResult and defines its propagation 
through And/Or.
   - Uses the first exact scalar index result. If none is available, scans the 
original candidate scope to recover matches that an inexact index may omit.
   - Passes eligible rows between index queries and pushes scalar predicates 
into fallback data reads.
   - Adds TopKBatchReader to select and order root search results across all 
files and supplied splits. Internal score columns are removed when not 
projected.
   - Normalizes dictionary-encoded columns before combining candidates, 
allowing files with different physical string encodings to participate in the 
same TopK.
   - Global TopK with fallback branches is rejected.
   - Root vector searches require an explicit distance type. 
   
   ### Tests
   Coverage includes:
   
   - Result exactness and And/Or behavior.
   - First-exact index selection and candidate propagation.
   - Timestamp precision loss, recovery of missing rows, and avoidance of 
unnecessary scalar rechecks.
   - Global TopK across files and splits, score projection, metric ordering, 
ties, batching, and deletion vectors.
   
   ### API and Format
   - Adds FileIndexResult::IsExact() and makes And/Or pure virtual. Custom 
subclasses must initialize exactness and implement both methods.
   - Extends bitmap/scored result construction with exactness.
   - Adds FileIndexReader::Evaluate(predicate, candidates).
   - Requires distance_type for root vector searches.
   - Defines root search limits and ordering across all splits supplied to 
TableRead::CreateReader.
   
   ### Documentation
   
   Updates the file-index guide and public API comments for result exactness.
   
   ### Generative AI tooling
   
   Generated-by: OpenAI Codex


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to