JingsongLi opened a new pull request, #10416: URL: https://github.com/apache/paimon/pull/10416
### Purpose Keep the existing `with_blob_consumer` API on NativeTableWrite. Installing a consumer currently invokes `__getattr__` and switches to the Python writer. Depends on https://github.com/apache/paimon-rust/pull/1063. This PR is a draft until that core/binding capability lands. Native CI continues to build paimon-rust `main`. - Forward consumers to the Rust batch/stream writer and decode serialized descriptors into the existing Python BlobDescriptor. - Preserve the original consumer when row-aware writes select the Python writer. - Keep callback timing, payload ranges, flush handling, copy behavior and ownership rules in Rust core, using Java semantics. - Exercise both Native and Python writers for scalar/ARRAY/MAP Blob callbacks, NULLs and empty values, referenced ranges, external paths, batch/stream writes, exception preservation, local flush visibility, callback replacement, partial projections and consumer-held files after abort/close/failure. - Change an existing abort regression to use the public `writer.abort()` API so it runs with either writer backend. ### Tests With the dependent Rust branch installed locally: - 499 passed in the Blob/collection/managed Blob/native write/update regression scope with Native planning, reads, writes, updates and commits enabled. Every Native execution counter was exercised. - Existing BlobConsumer regression suite without Native overrides: 6 passed. - Changed-file flake8 using `dev/cfg.ini` and diff checks passed. - Rust core unit suite: 3758 passed; full Rust Python binding suite: 326 passed. The new tests verify that callback-enabled Arrow writes retain NativeTableWrite and do not switch to the Python writer. No new Python public API or dependency. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
