mbutrovich commented on code in PR #11174:
URL: https://github.com/apache/arrow-rs/pull/11174#discussion_r4325738290
##########
arrow-select/src/coalesce.rs:
##########
@@ -853,6 +991,23 @@ trait InProgressArray: std::fmt::Debug + Send + Sync {
}
}
+ /// Copy rows at `indices` from `source` into the in-progress array.
+ /// `indices` must already be bounds-checked against `source`.
+ fn copy_rows_by_indices(
+ &mut self,
+ source: &ArrayRef,
+ indices: &UInt32Array,
+ ) -> Result<(), ArrowError> {
+ if indices.is_empty() {
+ return Ok(());
+ }
+ let taken = take(source.as_ref(), indices, None)?;
+ self.set_source(Some(taken));
+ let result = self.copy_rows(0, indices.len());
+ self.set_source(None);
+ result
Review Comment:
With the `has_non_specialized_filter_columns` check gone, this default now
runs for sparse selections on `Utf8`, `Boolean`, `FixedSizeBinary`,
`Dictionary`, `List`, and the other column types without a specialized
`copy_rows_by_indices`. The tests that reach `push_batch_with_index` use only
`Int32` and `Utf8View` columns, so nothing checks this path yet. Could we add a
test with a mixed schema, for example `Int32`, `Utf8`, `Boolean`, and
`FixedSizeBinary` with nulls, that splits across `target_batch_size` and
compares the output against `take_record_batch`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
