JingsongLi opened a new pull request, #1494: URL: https://github.com/apache/paimon-rust/pull/1494
### Purpose Part of #1490. Port the joint outer and inner probe selection from apache/paimon#11249. Depends on #1491. [Review the joint probing diff](https://github.com/JingsongLi/paimon-rust/compare/59a0dee1...6bd91436). ### Brief change log - Start with four outer shards and sixteen local lists for the calibrated unfiltered L2 IVF-FLAT and IVF-PQ layout. Confirm the shard sizes and local list counts before selecting that policy. - For older descriptors, check the native header and evaluate both usable query widths on the same open reader. Each query keeps the results of its selected policy. - Retry a short local joint search with the ordinary inner width. Widen the remaining outer search with the ordinary policy if Top-K is still underfilled. ### Tests The regressions exercise the exact local probe boundary, ordinary retries, mixed batch routes, format compatibility with and without persisted geometry, and the switch back to independent probing. ### API and Format The policy uses existing query options, manifest descriptors and index files. Explicit probe widths and filter settings retain independent sizing. ### Documentation Describes the calibrated layout and the compatibility path for older shard descriptors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
