juntaozhang opened a new pull request, #9389:
URL: https://github.com/apache/paimon/pull/9389
### Purpose
This PR introduces `VariantShreddingTypePruner`, which builds a path trie
from the logical Variant projection and recursively prunes unneeded typed
columns while preserving `value` fallbacks for paths that cannot be satisfied
from typed columns. Supported cases include:
- Nested object field pruning, e.g. `$.a.b`.
- Nested fields inside array elements, e.g. `$.arr[0].x`.
- Top-level array element pruning, e.g. `$[0].x`.
- Case-insensitive field matching with duplicate-field detection.
- Canonical three-level Parquet lists.
Two-level or non-canonical list layouts and map-typed `typed_value` groups
are kept unchanged as a safe fallback.
### Tests
- `VariantShreddingReadTest#testReadNestedVariantWithPruning`: end-to-end
read test for both shredded and unshredded Parquet file, covering nested object
fields, array index paths, missing fields inside array elements, and reading
whole array elements as `VARIANT`.
- `VariantShreddingTypePrunerTest`: unit tests covering
- nested object field pruning,
- missing top-level / nested field fallback to `value`,
- mixed existing and missing fields,
- whole-field `keepAll` semantics and so on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
