JingsongLi opened a new pull request, #10642: URL: https://github.com/apache/paimon/pull/10642
## Dependency Depends on https://github.com/apache/paimon-rust/pull/1165. Keep this draft until that PR is merged. Native CI continues building Rust main; no compatibility probes, fallback for missing new methods, or CI branch override are added. ## Summary Complete the four remaining native write/update routes: - Preserve the already-bound FUSE mount for writes, REST commits and copied branches. Bind the provider before historical schema resolution and cache the bound native read view. Keep custom FileIO implementations on their existing path. - Route batch truncate-table and truncate-partitions through the native REST committer. Preserve Java's lifecycle: truncate-table is one-shot, truncate-partitions does not consume the batch guard, and an earlier explicit abort can safely switch the cached committer from stream to batch mode. Mutation failures never trigger Python retries or automatic CommitMessage deletion. - Use Rust's automatic postpone fixed-bucket planning when no explicit plan/default is supplied. Align Python's existing in-process planner with current Java batch rules: infer from current input only and round up to a power of two before applying the cap. Keep known layouts, and preserve deferred Python Blob sources until core consumes them. - Support callable SET/INSERT values in ordinary MERGE through the existing core assignment machinery. Public callbacks consume qualified t./s. columns, retain complete input fields, restore coalesced ON-key aliases with consistent types, validate Arrow result lengths/types, and propagate original callback errors. Existing predicate/self-merge adapters retain their explicit read_columns projection. The initial capability inventory overstated callable support: ordinary MERGE previously rejected these values; this PR adds that explicitly requested API to both local and Ray execution paths. Matching, casting and staging for the native path remain in Rust core. Byte-based bucket inference uses the existing Python BinaryRow estimate. Spark's newer staged-file/rescale executor is outside the Python capabilities being completed here; identical compressed-file-based bucket counts are not promised. ## Verification - Focused write/commit/read-view/MERGE/postpone Python suites: **161 passed, 143 skipped, 12 subtests passed**. - Changed Python files: flake8 with repository configuration passed. - Three review rounds; fixed committer mode/lifecycle, provider binding order, callable projection/key-type, TIME offset and FLOAT precision issues. - Added real native integration tests for FUSE main/copied-branch publication and truncate, Java typed partition specs, complete-clause callable results/errors, automatic postpone counts and Blob lifetime. These skip locally because no Native extension is installed; no extension was rebuilt or installed. - Ray integration tests could not run locally because Ray is absent. Updated its old callable-rejection tests and added positive callback coverage for CI. - Companion Rust core: 4258 tests passed, 6 ignored; core and Python binding Clippy passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
