Cintu07 opened a new pull request, #11070:
URL: https://github.com/apache/arrow-rs/pull/11070

   ### Which issue does this PR close?
   
   - Closes #11059.
   
   ### Rationale for this change
   
   FixedSizeListArray::value_offset casts a usize offset to i32, so once a row 
starts past i32::MAX child values it returns a negative number. the avro 
FixedSizeList encoder cast that back to usize and panicked writing the batch.
   
   ### What changes are included in this PR?
   
   - deprecate value_offset in favour of i * value_length() as usize, same as 
FixedSizeBinaryArray::value_offset in #9910, and panic instead of wrapping
   - the avro encoder uses idx * elem_len instead
   - take_value_indices_from_fixed_size_list computes the offset in usize and 
returns an error when it doesn't fit u32
   - the value_offset asserts come out of the tests, and the slice test checks 
row values instead
   
   ### Are these changes tested?
   
   yes. fixed_size_list_encoder_int32_sliced is new and covers the avro change. 
the overflow itself needs 384 MiB so it isn't a unit test, but the repro from 
the issue panics on 59.3.0 and writes the batch on this branch. arrow-array 
972, arrow-select 446 and arrow-avro 515 tests pass, and clippy is clean.
   
   ### Are there any user-facing changes?
   
   FixedSizeListArray::value_offset is deprecated and panics past i32::MAX. 
take on a FixedSizeList returns an error past u32::MAX instead of wrapping.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to