ahlag commented on code in PR #11661:
URL: https://github.com/apache/arrow-rs/pull/11661#discussion_r4370941372
##########
arrow-array/src/ffi.rs:
##########
@@ -293,7 +293,19 @@ impl ImportedArrowArray<'_> {
};
let data_layout = layout(&self.data_type);
- let buffers = self.buffers(data_layout.can_contain_null_mask,
data_layout.variadic)?;
+ let mut buffers = self.buffers(data_layout.can_contain_null_mask,
data_layout.variadic)?;
+
+ // The values buffer of an empty array at offset 0 is imported as
empty (see
+ // `buffer_len`), so its lone offset, which can be any value, is reset
to 0.
+ if len == 0
+ && offset == 0
+ && matches!(
+ self.data_type,
+ DataType::Utf8 | DataType::Binary | DataType::LargeUtf8 |
DataType::LargeBinary
Review Comment:
Thanks @Jefffrey, good question, I hadn't checked those!
I don't think they're affected. The problem here comes from the `buffer_len`
shortcut for `Utf8`/`Binary`: for an empty array at offset 0 the importer
itself drops the values buffer to empty, so the lone offset ends up pointing
past it. For `List`/`LargeList`/`Map` the values are a child array that's
imported with its own length, so nothing gets cut and the offset still fits.
E.g. `slice(1, 0)` of a list exports offsets `[2]` with a child of length 3,
and that's what comes back.
To be sure I added `test_empty_slice_of_lists` in bfcd5ba, which round-trips
`slice(1, 0)` of each through `to_ffi`/`from_ffi` and checks `validate_full`
and equality. It passes with and without the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
