gene-db commented on code in PR #461:
URL: https://github.com/apache/parquet-format/pull/461#discussion_r1837129449
##########
VariantEncoding.md:
##########
@@ -416,14 +444,36 @@ Field names are case-sensitive.
Field names are required to be unique for each object.
It is an error for an object to contain two fields with the same name, whether
or not they have distinct dictionary IDs.
-# Versions and extensions
+## Versions and extensions
An implementation is not expected to parse a Variant value whose metadata
version is higher than the version supported by the implementation.
However, new types may be added to the specification without incrementing the
version ID.
In such a situation, an implementation should be able to read the rest of the
Variant value if desired.
-# Shredding
+## Shredding
A single Variant object may have poor read performance when only a small
subset of fields are needed.
A better approach is to create separate columns for individual fields,
referred to as shredding or subcolumnarization.
[VariantShredding.md](VariantShredding.md) describes the Variant shredding
specification in Parquet.
+
+## Conversion to JSON
+
+Values stored in the Variant encoding are a superset of JSON values.
+For example, a Variant value can be a date that has no equivalent type in JSON.
+To maximize compatibility with readers that can process JSON but not Variant,
the following conversions should be used when producing JSON from a Variant:
+
+| Variant type | JSON type | Representation requirements
| Example |
+|---------------|-----------|----------------------------------------------------------|--------------------------------------|
+| Null type | null | `null`
| `null` |
+| Boolean | boolean | `true` or `false`
| `true` |
+| Exact Numeric | number | Digits in fraction must match scale, no exponent
| `34`, 34.00 |
Review Comment:
@rdblue I am confused with this JSON chart then. If we are talking about
"storage implementation", then are you expecting there is a "storage
implementation" that is converting variant values to JSON? When will storage
convert a variant value to a JSON string?
I originally thought this chart was trying to say, "When an engine wants to
convert a variant value to a JSON string, here are the rules". Therefore, we
should allow engines to cast integral decimals to integers before converting to
JSON, as you already mentioned in your previous comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
