Rich-T-kid commented on code in PR #10945:
URL: https://github.com/apache/arrow-rs/pull/10945#discussion_r3919921334
##########
arrow-select/src/take.rs:
##########
@@ -1581,12 +1594,43 @@ pub fn take_record_batch(
record_batch: &RecordBatch,
indices: &dyn Array,
) -> Result<RecordBatch, ArrowError> {
- let columns = record_batch
+ /*let columns = record_batch
.columns()
.iter()
.map(|c| take(c, indices, None))
.collect::<Result<Vec<_>, _>>()?;
- RecordBatch::try_new(record_batch.schema(), columns)
+ RecordBatch::try_new(record_batch.schema(), columns)*/
+ unsafe { take_record_batch_unchecked(record_batch, indices) }
+}
+
+/// Take rows by index from [`RecordBatch`], returning a new [`RecordBatch`],
without bounds
+/// checking.
+///
+/// # Safety
+///
+/// The caller must guarantee that every non-null value in `indices` is a
valid row index for
+/// `record_batch` (i.e. `index < record_batch.num_rows()`). Violating this
will cause a panic
+/// or undefined behaviour inside the inner kernels.
+///
+/// # Errors
+///
+/// Returns an [`ArrowError`] if `indices` is not an integer array type.
+pub unsafe fn take_record_batch_unchecked(
+ record_batch: &RecordBatch,
+ indices: &dyn Array,
+) -> Result<RecordBatch, ArrowError> {
+ downcast_integer_array!(
Review Comment:
the TLDR is that check_bounds refers to
```rust
fn check_bounds<T: ArrowPrimitiveType>(
len: usize,
indices: &PrimitiveArray<T>,
) -> Result<(), ArrowError>
```
it checks that all the indices are valid within the arrays so something like
```rust
let int_values = int32Array::from([1,2,3,4,5]);
let indices = int32Array::from([1,20,200]);
take(int_values,indices);
```
fails before any of the `take_impl` branches run.
thats what I thought the linked issue
(https://github.com/apache/arrow-rs/issues/8879) was referring to but, skipping
this check is as simple as `take(array,None)` which skips the bounds checks.
PRs before this one threaded a const generic through the `take_impl`
branches, enabling code paths that use unsafe accessor methods to grab array
values directly, which should be slightly faster. These paths only get used
with `take_impl<_, false>`, since that pushes the safety guarantees onto the
caller. The code's also structured so hot loops don't recheck that value on
every iteration, since that would probably cost more than it saves.
Then there's this PR, which adds the final piece: threading all the variants
together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
