lxy-9602 opened a new pull request, #8392: URL: https://github.com/apache/paimon/pull/8392
## What is changed This PR adds append-only read/write support for MAP shared-shredding layout. MAP fields configured with `map.storage-layout=shared-shredding` are written with a physical shredded layout and per-field metadata in ORC/Parquet footers, and are reconstructed back to logical MAP values during reads. The read path is also refactored to use a unified shredding read-plan model for both Variant and MAP shared-shredding. ORC/Parquet readers now create a `ShreddingReadPlan` from file metadata/schema, read the corresponding physical row type, and wrap the raw reader with a common `ShreddingFormatReader` to assemble physical batches back into logical batches. Variant and MAP shared-shredding therefore share the same high-level read flow instead of having separate format-specific conversion models. ## Supported scope - Append-only tables. - ORC and Parquet data files. - MAP shared-shredding write path with footer metadata. - MAP shared-shredding read path with schema evolution/projection. - Variant read materialization moved into the shared shredding read-plan framework. ## Limitations Rewrite/compaction paths are not supported for MAP shared-shredding yet. Unsupported table modes and incompatible combinations, such as primary-key tables or combining Variant with MAP shared-shredding, are rejected by validation or fail fast in write paths. ## Tests Added coverage for MAP shared-shredding metadata, write plans, read plans, append table end-to-end read/write, projection, null values, overflow values, adaptive column restoration, layout switching, data evolution interaction, and validation of unsupported configurations. Existing Variant shredding tests continue to cover the refactored Variant read path. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
