klion26 commented on code in PR #9599:
URL: https://github.com/apache/arrow-rs/pull/9599#discussion_r2973454162
##########
parquet-variant-compute/src/shred_variant.rs:
##########
@@ -949,6 +984,54 @@ mod tests {
}
}
+ #[test]
+ fn test_append_null_mode_semantics() {
+ let cast_options = arrow::compute::CastOptions::default();
Review Comment:
Maybe we also need to add tests for
`VariantToShreddedVariantRowBuilder::{Array, Object}`
##########
parquet-variant-compute/src/variant_to_arrow.rs:
##########
@@ -941,7 +942,14 @@ where
match value {
Variant::List(list) => {
for element in list.iter() {
- self.element_builder.append_value(element)?;
+ match element {
Review Comment:
Why do we need to change this?
##########
parquet-variant-compute/src/shred_variant.rs:
##########
@@ -102,19 +102,49 @@ pub fn shred_variant(array: &VariantArray, as_type:
&DataType) -> Result<Variant
))
}
+/// Controls how `append_null` is encoded for a shredded `(value,
typed_value)` pair.
+///
+/// | Mode | Struct validity bit | `value` | `typed_value` | Meaning |
+/// | --- | --- | --- | --- | --- |
+/// | `TopLevelVariant` | null | NULL | NULL | SQL NULL at the top-level
variant row |
+/// | `ObjectField` | non-null | NULL | NULL | Missing object field |
+/// | `ArrayElement` | non-null | `Variant::Null` | NULL | Explicit null array
element |
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum AppendNullMode {
+ TopLevelVariant,
+ ObjectField,
+ ArrayElement,
+}
+
+impl AppendNullMode {
+ fn append_to_struct_nulls(self, nulls: &mut NullBufferBuilder) {
Review Comment:
The name `append_to_struct_nulls` is not easy to get the meaning for me, but
I don't have a better name for now.
##########
parquet-variant-compute/src/shred_variant.rs:
##########
@@ -204,33 +235,31 @@ impl<'a> VariantToShreddedVariantRowBuilder<'a> {
}
}
-/// A top-level variant shredder -- appending NULL produces typed_value=NULL
and value=Variant::Null
+/// A shredded primitive field builder.
pub(crate) struct VariantToShreddedPrimitiveVariantRowBuilder<'a> {
value_builder: VariantValueArrayBuilder,
typed_value_builder: PrimitiveVariantToArrowRowBuilder<'a>,
nulls: NullBufferBuilder,
- top_level: bool,
+ null_mode: AppendNullMode,
Review Comment:
Is it a reasonable solution to move `nulls` into `null_mode`?
##########
parquet-variant-compute/src/shred_variant.rs:
##########
@@ -102,19 +102,49 @@ pub fn shred_variant(array: &VariantArray, as_type:
&DataType) -> Result<Variant
))
}
+/// Controls how `append_null` is encoded for a shredded `(value,
typed_value)` pair.
+///
+/// | Mode | Struct validity bit | `value` | `typed_value` | Meaning |
+/// | --- | --- | --- | --- | --- |
+/// | `TopLevelVariant` | null | NULL | NULL | SQL NULL at the top-level
variant row |
+/// | `ObjectField` | non-null | NULL | NULL | Missing object field |
+/// | `ArrayElement` | non-null | `Variant::Null` | NULL | Explicit null array
element |
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum AppendNullMode {
+ TopLevelVariant,
+ ObjectField,
+ ArrayElement,
+}
+
+impl AppendNullMode {
+ fn append_to_struct_nulls(self, nulls: &mut NullBufferBuilder) {
Review Comment:
As we have extracted the AppendNullMode, do we need to unify these two
function calls into a single one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
