JingsongLi opened a new pull request, #459: URL: https://github.com/apache/paimon-rust/pull/459
## Summary Adds Variant shredding support aligned with Java, including configured and inferred shredding schemas. The format-level shredding read/write logic is kept reusable so future formats can plug in a physical writer factory instead of duplicating the state machine. ## Changes - Add Variant shredding schema construction, inference, cast, and rebuild utilities. - Add Arrow helpers for configured/inferred physical fields, logical-to-physical batch conversion, and shredded Variant reassembly on read. - Add reusable `ShreddingFormatWriter` / `ShreddingFormatReader` plus a physical writer factory abstraction. - Keep Parquet focused on raw Parquet IO and a `ParquetPhysicalWriterFactory`; route table format options into `DataFileWriter` through its constructor. - Add Parquet roundtrip coverage for configured and inferred Variant shredding schemas. ## Testing - [x] `cargo test -p paimon --lib` - [x] `cargo check -p paimon --features mosaic,vortex --lib` - [x] `cargo fmt --check` - [x] `git diff --check` ## Notes No migration is required. Existing non-shredded Variant files continue to read through the no-op path unless a shredded physical shape is detected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
