gene-db commented on code in PR #461:
URL: https://github.com/apache/parquet-format/pull/461#discussion_r1830065078
##########
VariantEncoding.md:
##########
@@ -416,14 +444,36 @@ Field names are case-sensitive.
Field names are required to be unique for each object.
It is an error for an object to contain two fields with the same name, whether
or not they have distinct dictionary IDs.
-# Versions and extensions
+## Versions and extensions
An implementation is not expected to parse a Variant value whose metadata
version is higher than the version supported by the implementation.
However, new types may be added to the specification without incrementing the
version ID.
In such a situation, an implementation should be able to read the rest of the
Variant value if desired.
-# Shredding
+## Shredding
A single Variant object may have poor read performance when only a small
subset of fields are needed.
A better approach is to create separate columns for individual fields,
referred to as shredding or subcolumnarization.
[VariantShredding.md](VariantShredding.md) describes the Variant shredding
specification in Parquet.
+
+## Conversion to JSON
+
+Values stored in the Variant encoding are a superset of JSON values.
+For example, a Variant value can be a date that has no equivalent type in JSON.
+To maximize compatibility with readers that can process JSON but not Variant,
the following conversions should be used when producing JSON from a Variant:
+
+| Variant type | JSON type | Representation requirements
| Example |
+|---------------|-----------|----------------------------------------------------------|--------------------------------------|
+| Null type | null | `null`
| `null` |
+| Boolean | boolean | `true` or `false`
| `true` |
+| Exact Numeric | number | Digits in fraction must match scale, no exponent
| `34`, 34.00 |
Review Comment:
Yes, the engine should be the one concerned with changing types.
However, my original question was about this JSON representation wording.
Currently, the `Representation requirements` for an `Exact Numeric` says the
`Digits in fraction must match scale`. However, because the `Exact Numeric` is
considered a logical type, the value `1` could be stored in the Variant as
`int8` 1 or `decimal(5,2)` 100. Both of those would be the same numeric value,
so we should allow truncating trailing zeros in the JSON representation,
instead of requiring that the digits in the fraction match the scale.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
