andygrove commented on code in PR #7511:
URL: https://github.com/apache/datafusion-comet/pull/7511#discussion_r4482656780
##########
native/shuffle/src/spark_unsafe/row.rs:
##########
@@ -831,237 +739,34 @@ fn append_struct_fields_field_major(
fields: &arrow::datatypes::Fields,
) -> Result<(), CometError> {
let num_rows = row_end - row_start;
- let num_fields = fields.len();
-
- // First pass: Build struct validity and collect which structs are null
- // We use a Vec<bool> for simplicity; could use a bitset for better memory
+ let mut struct_addresses = Vec::with_capacity(num_rows);
+ let mut struct_sizes = Vec::with_capacity(num_rows);
let mut struct_is_null = Vec::with_capacity(num_rows);
for i in row_start..row_end {
read_row_at!(parent_row, row_addresses_ptr, row_sizes_ptr, i);
-
let is_null = parent_row.is_null_at(column_idx);
struct_is_null.push(is_null);
if is_null {
struct_builder.append_null();
+ struct_addresses.push(0);
Review Comment:
The `0` entries for null structs are placeholders that are never read,
because every access is gated on `struct_is_null`. Would a short comment make
that clear for the next reader?
```suggestion
// Placeholders for a null struct. Never dereferenced, since
every read is gated on `struct_is_null`.
struct_addresses.push(0);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
