JingsongLi opened a new pull request, #1097: URL: https://github.com/apache/paimon-rust/pull/1097
## Summary Complete the Rust execution needed to move PyPaimon's existing local hybrid search into one native operation. - Apply accumulated data and partition predicates to each vector/full-text route before its own Top-K, reusing the existing route readers and their Java coverage/refinement semantics. - Resolve one snapshot for the operation and preserve the Snapshot object, including retained tags whose original snapshot file has expired. Accept an already resolved snapshot or an explicitly empty read view from the caller. - Execute at most four routes concurrently and retain route order for Java float fusion. Reuse the existing RRF, weighted-score and MRR rankers, route weights, route limits and final limit. - Use the full-text DSL API used by Java hybrid routes and report invalid configuration before an empty result, including unknown vector columns and zero final limits. - Add thin Python bindings and type stubs. The binding forwards configuration, predicates and snapshot JSON to core; planning, route execution and fusion remain in Rust core. - Reuse the DataFusion Arrow schema conversion when materializing hybrid results so text columns match the declared Utf8View output. Update SQL regressions to require an actual full-text definition, as Java does. The implementation follows [HybridSearchBuilderImpl](https://github.com/apache/paimon/blob/master/paimon-core/src/main/java/org/apache/paimon/table/source/HybridSearchBuilderImpl.java), [HybridSearchRanker](https://github.com/apache/paimon/blob/master/paimon-common/src/main/java/org/apache/paimon/globalindex/HybridSearchRanker.java) and the existing Java vector/full-text route readers. ## Verification - `cargo test --locked -p paimon --lib --features fulltext hybrid_search_builder`: 20 passed. - `cargo test --locked -p paimon-datafusion --features fulltext --test read_tables hybrid_search`: 5 passed. - `cargo +1.98.0 clippy --locked -p paimon -p paimon-datafusion -p pypaimon_rust --all-targets --features fulltext,vortex -- -D warnings`: passed. - `cargo check --locked -p paimon --lib` without fulltext: passed. - `cargo fmt --all -- --check` and `git diff --check`: passed. - Rebuilt the Python extension with maturin and ran the paired PyPaimon REST suite with native planning, reading, writing, updating and committing enabled: 73 passed, 3 subtests passed. Regressions cover real vector/full-text/scalar indexes, partial coverage, filtering before Top-K, all three rankers, more routes than the worker bound, authoritative REST snapshots and errors, fixed empty views, retained tags, malformed DSL, unsupported literals and query authorization. Two independent review passes found and fixed the missing-column and caller-pinned-view issues. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
