JingsongLi opened a new pull request, #10305: URL: https://github.com/apache/paimon/pull/10305
### Purpose Enable scalar BLOB Arrow writes through PyPaimon Native and verify that the resulting files remain interoperable with Python readers and committers. ### Changes - Open the native write route for top-level scalar BLOB fields. ARRAY/MAP BLOB, video and optional row-sidecar routes retain their existing Python implementation. - Select the Python writer on the first `write_row` for a BLOB table, preserving custom Blob streams and URI readers without materialization. Switching from already-started native Arrow writes remains rejected and is documented. - Align Python inline descriptor validation with Java's known-descriptor prefix parser, including valid v1/v2 payloads with trailing bytes. - Exercise normal/Blob roll grouping, file metadata, write-cols optimization, stream reuse, external paths and both committers. Assert native execution and cross-read every result with Python and Native readers/planners. - Add cleanup, streaming-row, inline descriptor and HTTP descriptor regressions, including gzip/deflate, unknown lengths, error cleanup and an 18 MiB copy requiring a single GET. - Make existing tests explicit about Python internals and unordered cross-partition scans; keep the deferred-reader limit tests supplied with both partitions. ### Dependency Requires https://github.com/apache/paimon-rust/pull/989. This PR is draft until that change is merged. CI continues installing apache/paimon-rust@main; no fork or branch pin is introduced. ### Verification Tests run with the locally rebuilt Rust binding, PyArrow 18 and all five Native CI options enabled. Flake8 and git diff checks pass. Expanded regression: **1,238 passed, 3 skipped, 67 subtests passed** across native writes, updates, nested updates, external paths, data directories, Blob read/write, deferred resolution, commit, write buffers and data evolution. Three optional Vortex cases were deselected because the local Vortex dependency is absent. Native execution counters: 1,044 plans, 1,425 reads, 937 writes and 7 commits; updates exercised row-id (84), grouped (53), predicate (57), upsert (34) and incremental (15) paths. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
