JingsongLi opened a new pull request, #981: URL: https://github.com/apache/paimon-rust/pull/981
### Purpose Follow up on #974 to honor `data-file.external-paths` when creating data files. Rust could resolve existing external files and place external deletion vectors, but its table writers still created data files under the table directory. This also kept PyPaimon's external-path tables on the Python writer. ### Brief change log - Add an internal, Java-style per-bucket data path factory used by append, primary-key, postpone, data-evolution and copy-on-write writers. - Share the path provider between normal and dedicated Blob files, and between primary-key data and input changelog files. Select each file's destination once, persist `external_path`, and place index sidecars beside that destination. - Honor round-robin, weight-robin, entropy-inject, specific-fs and none. Match Java's weight validation and disabled-strategy handling. - Revalidate the existing primary-key managed Blob restriction when creating a writer so copied table options cannot bypass it. - Retain successful physical-column outputs until all closes complete, then clean up all data and sidecar files if any close fails. ### Tests 31 focused Rust tests passed: - `external_data_write_test` (5 tests, including the writer/strategy matrix) - `data_file_directory_test` (7 tests) - `table_update_paths_test` (7 tests) - `table::external_path::tests` (8 tests) - `table::data_file_writer::tests` (3 tests) - `table::dedicated_format_file_writer::tests` (1 failure-injection test) Also passed: - `cargo fmt --all --check` - `cargo clippy --locked --all-targets --workspace --features fulltext,vortex -- -D warnings` - Built the Python bindings and ran the companion PyPaimon regression with all five native test flags: 1593 passed, 2 skipped, 74 subtests passed. This includes append/PK/data-evolution interoperability, batch and stream writes, escaped partitions, destination changes, updates/upserts/deletes, historical reads and abort cleanup. ### API and Format No public API or storage-format change. File placement uses the existing Java-compatible options and `DataFileMeta.external_path` field. ### Documentation The companion PyPaimon change documents native external writes. PyPaimon's Blob writer fallback is preserved because its remaining native Blob input/metadata behavior needs separate work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
