JingsongLi opened a new pull request, #1065: URL: https://github.com/apache/paimon-rust/pull/1065
### Purpose Linked issue: close #1064 Keep the existing PyPaimon `with_blob_uri_reader_factory` capability on native Arrow writes. Today that setter switches to the Python writer, and Rust core cannot copy Blob references through a caller-provided URI reader. The interfaces and lifecycle follow Java `UriReaderFactory`, `UriReader`, `ReusingBlobRefStreamProvider`, and `PrimaryKeyBlobExternalizer`. ### Brief change log - Add public core `UriReaderFactory`, `UriReader` and `UriInputStream` interfaces and a private bounded-source reuse helper. - Copy exact descriptor windows with bounded buffers, accept successful short reads, reject premature EOF and oversized reads, and copy unknown-length sources to EOF on independent streams. - Reuse reader/URI sources, reopen after failed repositioning, and close sources and destinations while preserving the first error. - Propagate the factory through append scalar/array/map Blob writers and existing primary-key managed Blob packs, including postpone writes. - Close custom sources before removing unprepared PK packs; reject late factory changes for routed postpone-fixed writes. - Add thin Python callback adapters, preserve original callback exceptions and reader identity across interleaved physical writers, and expose the existing setter for batch and stream writes. ### Tests - `cargo test --locked -p paimon --lib`: 3,773 passed, 6 ignored. - Full Python binding suite against the prepared shared warehouse: 331 passed. - PyPaimon focused integration tests: 48 passed. - PyPaimon extended Blob/write/update suite with all five Native switches enabled: 518 passed (659 native plans, 863 native reads, 800 native writes, 169 native commits, all update categories exercised). - `cargo +1.98.0 clippy --locked --all-targets --workspace --features fulltext,vortex -- -D warnings`: passed. - Rust fmt, Python lint, dependency report verification and all 21 generated release legal files: passed. - Regression tests reproduced the pre-fix reader-identity, PK source-close and postpone-fixed late-setter failures. ### API and Format Adds injectable Blob source-reader interfaces in core and `with_blob_uri_reader_factory(factory)` in Python batch/stream writers. Passing `None` restores the default before writing. No storage format changes and no new transitive dependency packages; `async-trait` and `bytes` were already in the Python dependency graph. ### Documentation Documented the source-stream contract and lifecycle in the core traits, Python stubs and binding README. A companion Apache Paimon Python PR will keep the existing PyPaimon setter on this native path and add end-to-end tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
