scovich commented on code in PR #8122:
URL: https://github.com/apache/arrow-rs/pull/8122#discussion_r2274544763
##########
parquet-variant-compute/src/variant_array.rs:
##########
@@ -192,7 +202,96 @@ impl VariantArray {
/// Return a reference to the metadata field of the [`StructArray`]
pub fn metadata_field(&self) -> &BinaryViewArray {
- self.shredding_state.metadata_field()
+ &self.metadata
+ }
+
+ /// Return a reference to the value field of the `StructArray`
+ pub fn value_field(&self) -> Option<&BinaryViewArray> {
+ self.shredding_state.value_field()
+ }
+
+ /// Return a reference to the typed_value field of the `StructArray`, if
present
+ pub fn typed_value_field(&self) -> Option<&ArrayRef> {
+ self.shredding_state.typed_value_field()
+ }
+}
+
+/// One shredded field of a partially or prefectly shredded variant. For
example, suppose the
+/// shredding schema for variant `v` treats it as an object with a single
field `a`, where `a` is
+/// itself a struct with the single field `b` of type INT. Then the physical
layout of the column
+/// is:
+///
+/// ```text
+/// v: VARIANT {
+/// metadata: BINARY,
+/// value: BINARY,
+/// typed_value: STRUCT {
+/// a: SHREDDED_VARIANT_FIELD {
+/// value: BINARY,
+/// typed_value: STRUCT {
+/// a: SHREDDED_VARIANT_FIELD {
+/// value: BINARY,
+/// typed_value: INT,
+/// },
+/// },
+/// },
+/// },
+/// }
+/// ```
+///
+/// In the above, each row of `v.value` is either a variant value (shredding
failed, `v` was not an
+/// object at all) or a variant object (partial shredding, `v` was an object
but included unexpected
+/// fields other than `a`), or is NULL (perfect shredding, `v` was an object
containing only the
+/// single expected field `a`).
+///
+/// A similar story unfolds for each `v.typed_value.a.value` -- a variant
value if shredding failed
+/// (`v:a` was not an object at all), or a variant object (`v:a` was an object
with unexpected
+/// additional fields), or NULL (`v:a` was an object containing only the
single expected field `b`).
+///
+/// Finally, `v.typed_value.a.typed_value.b.value` is either NULL (`v:a.b` was
an integer) or else a
+/// variant value.
+pub struct ShreddedVariantFieldArray {
Review Comment:
Another thought: The variant shredding spec doesn't name the type of the
group that represents shredded object fields, but it's definitely _NOT_
VARIANT. It's some other thing that must never have a `metadata` column and
which must always nest inside a VARIANT type that provides the `metadata`
column it depends on. And it's `required` where VARIANT is `optional`. And it
allows both `value` and `typed_value` to be NULL/missing where VARIANT forbids
that combination.
My worry is, the two are really not the same thing -- other than they happen
to contain two fields with the same name -- and if we try to conflate the two
we're going to cause a lot of trouble for ourselves.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
