mbutrovich commented on code in PR #12252:
URL: https://github.com/apache/arrow-rs/pull/12252#discussion_r4605963629
##########
arrow-select/src/coalesce/byte_view.rs:
##########
@@ -502,10 +503,8 @@ impl<B: ByteViewType> InProgressByteViewArray<B> {
.get_unchecked(source_buffer_index)
.get_unchecked(source_buffer_offset..source_buffer_offset + length)
};
- view.offset = dst_buffer
- .len()
- .try_into()
- .expect("byte view offset exceeds u32::MAX");
+ // The final buffer length is bounded by `max_buffer_size`.
+ view.offset = dst_buffer.len() as u32;
Review Comment:
This offset fits in `u32` only if `selected_bytes` is exact, since that is
what keeps `dst_buffer.len()` within `max_buffer_size`. Could we add a
`debug_assert!` for it, so tests and Miri catch a caller that gets
`selected_bytes` wrong? The comment could also say why the bound makes the cast
safe.
```suggestion
// `selected_bytes` keeps the buffer within
`max_buffer_size`, which
// is at most `i32::MAX`, so the offset fits in `u32`.
debug_assert!(dst_buffer.len() + length <= max_buffer_size);
view.offset = dst_buffer.len() as u32;
```
##########
arrow-select/src/coalesce/byte_view.rs:
##########
@@ -443,6 +443,7 @@ impl<B: ByteViewType> InProgressByteViewArray<B> {
}
return;
}
+ let max_buffer_size = max_buffer_size.min(i32::MAX as usize);
Review Comment:
With this cap, the `as u32` conversions below depend on `max_buffer_size <=
i32::MAX`. Before, the `expect` turned a wrong offset into a panic. Now a wrong
offset goes into the view, and `finish` builds the array with
[`new_unchecked`](https://github.com/apache/arrow-rs/blob/4495f6bd736086634405b8af6ed1f633c9065e2b/arrow-select/src/coalesce/byte_view.rs#L935),
so the bound is part of the safety contract.
Could we write it into the docs?
`append_indexed_views_and_copy_strings_split` converts with `as u32` too, but
it gets `max_buffer_size` from this function, and its [`# Safety`
section](https://github.com/apache/arrow-rs/blob/4495f6bd736086634405b8af6ed1f633c9065e2b/arrow-select/src/coalesce/byte_view.rs#L520-L527)
doesn't require the bound. I'd add "`max_buffer_size` must be at most
`i32::MAX`" there. The [doc comment on this
function](https://github.com/apache/arrow-rs/blob/4495f6bd736086634405b8af6ed1f633c9065e2b/arrow-select/src/coalesce/byte_view.rs#L419-L424)
should also say that `max_buffer_size` is capped at `i32::MAX`.
##########
arrow-select/src/coalesce/byte_view.rs:
##########
@@ -574,10 +573,7 @@ impl<B: ByteViewType> InProgressByteViewArray<B> {
.get_unchecked(buffer_index)
.get_unchecked(buffer_offset..buffer_offset + str_len)
};
- view.offset = dst_buffer
- .len()
- .try_into()
- .expect("byte view offset exceeds u32::MAX");
+ view.offset = dst_buffer.len() as u32;
Review Comment:
This cast has no comment. Could we add the same comment and `debug_assert!`
here? After the buffer switch above, `dst_buffer.len() + str_len <=
max_buffer_size` holds on both branches.
```suggestion
// The buffer stays within `max_buffer_size`, which is at
most
// `i32::MAX`, so the offset fits in `u32`.
debug_assert!(dst_buffer.len() + str_len <= max_buffer_size);
view.offset = dst_buffer.len() as u32;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]