JingsongLi opened a new pull request, #1493:
URL: https://github.com/apache/paimon-rust/pull/1493

   ### Purpose
   
   Part of #1490. Add Java-compatible scalar Bitmap filters to centroid shards, 
including cross-column OR. Aligns with apache/paimon#11306 and 
apache/paimon#11329.
   
   Depends on #1491. [Review the Bitmap-only 
diff](https://github.com/JingsongLi/paimon-rust/compare/59a0dee1...446b0796).
   
   ### Brief change log
   
   - Accept `extra_fields` on centroid builds. Write a Bitmap file per 
populated vector shard using row labels relative to the shared source range. 
The directory records the build schema and section lengths.
   - Bind exact `=`, `IN`, null checks, `AND`, and `OR` against compatible 
build fields. Intersect the result with row and deletion filters before ANN, 
and skip the ANN file for empty matches.
   - Use the existing scalar path for unsupported predicates or incompatible 
schemas. Keep vector and Bitmap files in one replaceable generation, addressed 
by the vector column.
   
   ### Tests
   
   Exercises scheduled exact prefiltering with an ANN file absent from a 
nonmatching shard, group replacement and drop, and a Java-generated two-section 
sparse Bitmap file.
   
   ### API and Format
   
   Uses Java's version 2 centroid metadata. Adds the optional `extra_fields` 
argument to DataFusion's create-index procedure and accepts the same JSON in 
Rust build options.
   
   ### Documentation
   
   Adds a centroid build example with two Bitmap columns and describes exact 
predicate support and fallback.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to