rdblue commented on code in PR #461:
URL: https://github.com/apache/parquet-format/pull/461#discussion_r1815653942
##########
VariantEncoding.md:
##########
@@ -39,13 +39,31 @@ Another motivation for the representation is that (aside
from metadata) each nes
For example, in a Variant containing an Array of Variant values, the
representation of an inner Variant value, when paired with the metadata of the
full variant, is itself a valid Variant.
This document describes the Variant Binary Encoding scheme.
-[VariantShredding.md](VariantShredding.md) describes the details of the
Variant shredding scheme.
+The [Variant Shredding spec](VariantShredding.md) describes the details of
shredding Variant values as typed Parquet columns.
+
+## Variant in Parquet
-# Variant in Parquet
A Variant value in Parquet is represented by a group with 2 fields, named
`value` and `metadata`.
-Both fields `value` and `metadata` are of type `binary`, and cannot be `null`.
+The Variant group must be annotated with the `VARIANT` logical type.
+Both fields `value` and `metadata` are of type `binary`.
+The `metadata` field is required and must be a valid Variant metadata, as
defined below.
+The `value` field is optional.
+When present, the `value` field must be a valid Variant value, as defined
below.
+The `value` field may be null only when parts of the Variant value are
shredded according to the Variant Shreedding spec.
+
+This is the expected representation in Parquet:
-# Metadata encoding
+```
+optional group variant_event (VARIANT) {
+ required binary metadata;
+ optional binary value;
Review Comment:
No. The encoding spec says that the value must be required when not
shredding. I didn't include that case here since this is documenting how to
shred variants.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
