0z5a opened a new pull request, #1018: URL: https://github.com/apache/arrow-nanoarrow/pull/1018
`c_array_shallow_copy()` reserves the extra buffers needed by a string view or binary view array by calling `ArrowArrayAddVariadicBuffers()`, but it calls it on `dst` (the not-yet-initialized output array) instead of on `tmp` (the array it is building). `dst` is usually uninitialized memory at that point, so this dereferences a garbage `private_data` pointer, and `tmp` never gets room for the variadic buffers, so the loop that follows writes past the end of its buffer arrays. Any string view or binary view array with at least one variadic data buffer goes through this path, i.e. any array with a value longer than 12 bytes, and every such array exported by pyarrow (which always includes a data buffer). The process segfaults or aborts, for example: ```python import nanoarrow as na array = na.c_array(["ab", None, "a string longer than 12 bytes"], na.string_view()) list(na.iterator.iter_py(array)) # crashes array[1:] # crashes ``` The same crash happens for `na.Array(pa.array([...], pa.string_view())).to_pylist()`, for exporting such an array to pyarrow via `__arrow_c_array__`, and when a view array is used as a child in `c_array_from_buffers()`. The branch was added in #808, so 0.8.0 and 0.9.0 are affected (I also see the crash with the 0.9.0 wheel). The existing iterator tests only use values of 12 bytes or fewer, which are inlined and give three buffers, so they never reach this branch. The fix passes `tmp` instead of `dst`. With it, the examples above return the same values as pyarrow's `to_pylist()`, and buffers are still released once the copies are dropped. I added a regression test that iterates and slices string view and binary view arrays with a non-inlined value; it crashes the test process without the fix and passes with it. The rest of the Python test suite passes. This change was drafted with Claude Code (claude-opus-5-5), which found the bug and generated the fix, the regression test and this description; the test results above are from the runs described. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
