alamb commented on code in PR #8839:
URL: https://github.com/apache/arrow-rs/pull/8839#discussion_r2560186355
##########
arrow-row/src/lib.rs:
##########
@@ -1762,6 +1907,110 @@ unsafe fn decode_column(
},
_ => unreachable!(),
},
+ Codec::Union(converters, null_rows, _mode) => {
+ let len = rows.len();
+
+ let DataType::Union(union_fields, mode) = &field.data_type else {
+ unreachable!()
+ };
+
+ let mut type_ids = Vec::with_capacity(len);
+ let mut rows_by_field: Vec<Vec<(usize, &[u8])>> = vec![Vec::new();
converters.len()];
+
+ for (idx, row) in rows.iter_mut().enumerate() {
+ // skip the null sentinel
+ let mut cursor = 1;
Review Comment:
> Hm, I thought union arrays don't support physical nulls. i.e.
I agree, https://arrow.apache.org/docs/format/Columnar.html#union-layout
says:
> Unlike other data types, unions do not have their own validity bitmap.
Instead, the nullness of each slot is determined exclusively by the child
arrays which are composed to create the union.
If the union itself doesn't have a null mask, is there any reason to include
a sentinel byte at all?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
