This is an automated email from the ASF dual-hosted git repository.
gabor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/parquet-format.git
The following commit(s) were added to refs/heads/master by this push:
new 3d8855a GH-486: Variant object shredding without field shredding
(#487)
3d8855a is described below
commit 3d8855afba64b2eca8d09828aa59885727efa85c
Author: Gabor Szadovszky <[email protected]>
AuthorDate: Thu Mar 6 08:19:54 2025 +0100
GH-486: Variant object shredding without field shredding (#487)
---
VariantShredding.md | 1 +
1 file changed, 1 insertion(+)
diff --git a/VariantShredding.md b/VariantShredding.md
index b3ecd4a..012601b 100644
--- a/VariantShredding.md
+++ b/VariantShredding.md
@@ -168,6 +168,7 @@ Readers can assume that a value is not an object if
`typed_value` is null and th
Each shredded field in the `typed_value` group is represented as a required
group that contains optional `value` and `typed_value` fields.
The `value` field stores the value as Variant-encoded `binary` when the
`typed_value` cannot represent the field.
This layout enables readers to skip data based on the field statistics for
`value` and `typed_value`.
+The `typed_value` field may be omitted when not shredding fields as a specific
type.
The `value` column of a partially shredded object must never contain fields
represented by the Parquet columns in `typed_value` (shredded fields).
Readers may always assume that data is written correctly and that shredded
fields in `typed_value` are not present in `value`.
