JingsongLi opened a new pull request, #10412: URL: https://github.com/apache/paimon/pull/10412
## Purpose Keep existing Daft grouped writes on Native. The public PyPaimon APIs already support precomputed buckets and coordinator-pinned dynamic indexes, but the Native wrapper previously switched to Python for those methods. Depends on https://github.com/apache/paimon-rust/pull/1056 merging first. The Native CI job continues to build Rust from `main`. ## Changes - Forward `with_dynamic_bucket_index` and `write_arrow_batch_to_bucket` to Rust. Reuse ordinary Native input normalization and selected-column projection, and retain the resolved index base when an advanced method selects Python before data is staged. - Extend Native CI instrumentation to count precomputed bucket writes. - Align the Python HASH maintainer with Java: publish only the complete new HASH file; the manifest combiner replaces the previous entry by partition and bucket. - Share Python RowKind filtering and its row selection with precomputed metadata normalization. Validate the entire partition group before notifying its maintainer, and notify every surviving key regardless of the upstream first-mapping hint. - Add Native batch/stream round trips, unsigned Java hashes, selected-column writes, snapshot restoration, checkpoint sequence, failure ownership, and actual Daft append/overwrite/filter regressions. Replace Python-specific automatic HASH DELETE assumptions with Java's bucket-owner/full-file replacement contract. This enables capabilities already supported by PyPaimon. The Rust core owns routing, index restoration, writing, and checkpoint state; the adapter does not move those operations into Python. ## Verification - Native write/Daft/schema/sequence suite: 246 passed, 1 skipped, 11 subtests passed; real native plans, reads, and writes were exercised. - Ordinary Python write/Daft suite: 111 passed, 10 skipped, 17 subtests passed, including schema-only writer objects and both RowKind filter settings. - Rust core table module: 1814 passed, 3 ignored; related integration tests: 26 passed. - Python binding suite: 317 passed, including four catalog smoke tests rerun after provisioning their local fixture. - Changed Python files pass the repository flake8 configuration; both repositories pass diff checks. - Independent review findings were fixed, with checkpoint-before-publication and filtered first-mapping regressions added. Kept as a draft until the Rust prerequisite merges into `main`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
