JingsongLi opened a new pull request, #10487: URL: https://github.com/apache/paimon/pull/10487
### Purpose Use the common Rust `Table.new_global_index_build_builder()` API instead of exposing sorted-index implementation details to PyPaimon. Depends on https://github.com/apache/paimon-rust/pull/1091. Keep this draft until the Rust API is on main. CI continues to install Rust main, with no fork or branch pin. - Migrate local native construction and existing sorted-index tests to the common builder. - Delegate eligible REST full-text builds to core, preserving the unpublished `CommitMessage` contract and Python commit API. - Normalize effective full-text options through the existing full-text converter, preserving list-valued stop words and boolean/string encodings. - Keep vector construction on the Python path: the existing Rust vector pipeline still requires dense, non-null vectors, while Java/Python preserve sparse relative row IDs when vectors are null. - Add REST tests that directly exercise the common Rust builder for full-text and all five vindex types, partition conjunction, per-build shard sizing, external placement, both committers, source conflicts with retained files, and reading Rust-created files via Python fast search. ### Tests Using the local Rust PR extension: `PYPAIMON_TEST_NATIVE_PLAN=1 .venv/bin/python -m pytest pypaimon/tests/native_generic_index_build_test.py pypaimon/tests/native_sorted_index_build_test.py pypaimon/tests/global_index_build_test.py -q` 153 passed, 19 subtests passed; 55 native plans exercised. The new regressions also compare native and original full-text metadata and searches for list/string stop words supplied through build or table options, and preserve nullable vector construction through the existing Python writer. Changed Python files pass Flake8 with `dev/cfg.ini`, Python 3.6 grammar parsing and `git diff --check`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
