0z5a opened a new pull request, #1018:
URL: https://github.com/apache/arrow-nanoarrow/pull/1018

   `c_array_shallow_copy()` reserves the extra buffers needed by a string view 
or binary view array by calling `ArrowArrayAddVariadicBuffers()`, but it calls 
it on `dst` (the not-yet-initialized output array) instead of on `tmp` (the 
array it is building). `dst` is usually uninitialized memory at that point, so 
this dereferences a garbage `private_data` pointer, and `tmp` never gets room 
for the variadic buffers, so the loop that follows writes past the end of its 
buffer arrays.
   
   Any string view or binary view array with at least one variadic data buffer 
goes through this path, i.e. any array with a value longer than 12 bytes, and 
every such array exported by pyarrow (which always includes a data buffer). The 
process segfaults or aborts, for example:
   
   ```python
   import nanoarrow as na
   
   array = na.c_array(["ab", None, "a string longer than 12 bytes"], 
na.string_view())
   list(na.iterator.iter_py(array))  # crashes
   array[1:]                         # crashes
   ```
   
   The same crash happens for `na.Array(pa.array([...], 
pa.string_view())).to_pylist()`, for exporting such an array to pyarrow via 
`__arrow_c_array__`, and when a view array is used as a child in 
`c_array_from_buffers()`. The branch was added in #808, so 0.8.0 and 0.9.0 are 
affected (I also see the crash with the 0.9.0 wheel). The existing iterator 
tests only use values of 12 bytes or fewer, which are inlined and give three 
buffers, so they never reach this branch.
   
   The fix passes `tmp` instead of `dst`. With it, the examples above return 
the same values as pyarrow's `to_pylist()`, and buffers are still released once 
the copies are dropped. I added a regression test that iterates and slices 
string view and binary view arrays with a non-inlined value; it crashes the 
test process without the fix and passes with it. The rest of the Python test 
suite passes.
   
   This change was drafted with Claude Code (claude-opus-5-5), which found the 
bug and generated the fix, the regression test and this description; the test 
results above are from the runs described.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to