Cintu07 opened a new issue, #11317:
URL: https://github.com/apache/arrow-rs/issues/11317
**Describe the bug**
concat_elements_binary_view_array and concat_elements_string_view_array work
out the data buffer size by adding the two element lengths. both come from
lengths(), which yields u32, so the add is u32 and a pair that sums past
u32::MAX wraps there. the check right after is data_size > i32::MAX and it sees
the wrapped number, so it passes. the builder then appends the real bytes and
make_view truncates the length with len as u32, so the element comes back empty.
**To Reproduce**
one element of 2 GiB concatenated with itself. the two lengths sum to
exactly u32::MAX + 1, which wraps to 0.
```rust
let len = 1_usize << 31;
let buffer: Buffer = MutableBuffer::from_len_zeroed(len).into();
let view = ByteView { length: len as u32, prefix: 0, buffer_index: 0,
offset: 0 }.as_u128();
let array = BinaryViewArray::try_new(ScalarBuffer::from(vec![view]),
vec![buffer], None).unwrap();
let out = concat_elements_binary_view_array(&array, &array).unwrap();
println!("{}", out.value(0).len());
```
prints 0. wants about 8 GiB and a release build, a debug build panics on the
add instead.
**Expected behavior**
an error. a view carries its length in 32 bits, so a value that long cannot
be represented at all, and the i32::MAX check already there is the right place
to say so.
**Additional context**
the fix is to add in usize. i have it with the repro as an ignored test, pr
coming right after this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
