JingsongLi opened a new pull request, #1494:
URL: https://github.com/apache/paimon-rust/pull/1494

   ### Purpose
   
   Part of #1490. Port the joint outer and inner probe selection from 
apache/paimon#11249.
   
   Depends on #1491. [Review the joint probing 
diff](https://github.com/JingsongLi/paimon-rust/compare/59a0dee1...6bd91436).
   
   ### Brief change log
   
   - Start with four outer shards and sixteen local lists for the calibrated 
unfiltered L2 IVF-FLAT and IVF-PQ layout. Confirm the shard sizes and local 
list counts before selecting that policy.
   - For older descriptors, check the native header and evaluate both usable 
query widths on the same open reader. Each query keeps the results of its 
selected policy.
   - Retry a short local joint search with the ordinary inner width. Widen the 
remaining outer search with the ordinary policy if Top-K is still underfilled.
   
   ### Tests
   
   The regressions exercise the exact local probe boundary, ordinary retries, 
mixed batch routes, format compatibility with and without persisted geometry, 
and the switch back to independent probing.
   
   ### API and Format
   
   The policy uses existing query options, manifest descriptors and index 
files. Explicit probe widths and filter settings retain independent sizing.
   
   ### Documentation
   
   Describes the calibrated layout and the compatibility path for older shard 
descriptors.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to