lszskye opened a new pull request, #8364:
URL: https://github.com/apache/paimon/pull/8364
## Purpose
Add read support for MAP shared-shredding data in `paimon-common`.
This change introduces a reader wrapper that rebuilds logical `MAP<STRING,
T>` values from shared-shredding physical ROW values.
## Changes
- Add `MapSharedShreddingReader`
- Wraps a physical `FileRecordReader<InternalRow>`.
- Lazily rebuilds shared-shredding MAP fields when
`InternalRow#getMap(pos)` is called.
- Handles overflow fields.
- Adds validation for invalid physical data:
- null `fieldMapping`
- null `fieldMapping` element
- unknown field id in metadata
- unknown overflow field id
- Add shared-shredding utility methods
- `getPhysicalColumnIndices`
- `isOverflowField`
- `buildSpecificPhysicalStructType`
## Limitation
This PR currently only supports reading the whole shared-shredding MAP field.
It does not yet support selecting / projecting specific MAP keys during
read. Because of this, rebuilt map entries currently follow the physical
metadata layout order instead of the user requested key order. A TODO is left
in the reader for future key-level projection support.
## Tests
Added unit coverage for:
- Logical MAP reconstruction from shared-shredding physical rows.
- Null logical MAP rows.
- Null MAP values retained in rebuilt results.
- Invalid `fieldMapping` and unknown field id handling.
- Overflow field reconstruction and validation.
- New `MapSharedShreddingUtils` helper methods.
## Verification
```shell
mvn -pl paimon-common -Pfast-build
-Dtest=MapSharedShreddingReaderTest,MapSharedShreddingUtilsTest test
git diff --check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
