This is an automated email from the ASF dual-hosted git repository.
AlenkaF pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/main by this push:
new 8c7b08f4269 GH-51041: [Python] Reject non-Buffer
FunctionOptions.deserialize input (#51129)
8c7b08f4269 is described below
commit 8c7b08f42694342ed1dbe7e8a351a947dae8e13d
Author: Stefan Wang <[email protected]>
AuthorDate: Thu Sep 10 01:15:53 2026 -0700
GH-51041: [Python] Reject non-Buffer FunctionOptions.deserialize input
(#51129)
### Rationale for this change
Calling `pyarrow.compute.FunctionOptions.deserialize()` with anything that
is not a `Buffer` crashes the interpreter. `None`, an `int`, a `list` and a
`bytes` object all segfault, so a caller who passes the wrong object, or who
forwards an unvalidated value, loses the process rather than seeing an
exception.
`deserialize()` hands its argument to `pyarrow_unwrap_buffer()`, which
returns a null pointer for a non-`Buffer`. The `deref()` on the next line then
dereferences null.
Before this change the call terminates the process. After it, the call
raises `TypeError` naming the type it received.
### What changes are included in this PR?
The static method now declares its parameter as `Buffer buf not None`,
which is what its docstring already says it takes. Cython rejects a wrong type
before the unwrap runs, so the null pointer is never produced.
### Are these changes tested?
Yes, by a parametrized case over the four inputs above.
<details>
<summary>Red, with this commit's source file reverted to its
parent</summary>
```console
$ git checkout HEAD~1 python/pyarrow/_compute.pyx
$ pip install -e python --no-build-isolation
$ python -X faulthandler -c "import pyarrow.compute as pc;
pc.FunctionOptions.deserialize(None)"
Fatal Python error: Segmentation fault
Current thread 0x00000001f6fbe180 (most recent call first):
File "<string>", line 1 in <module>
Extension modules: numpy._core._multiarray_umath,
numpy.linalg._umath_linalg, pyarrow.lib, pyarrow._compute (total: 4)
```
The same crash occurs for `1`, `[]` and `b''`.
</details>
<details>
<summary>Green, with the fix applied</summary>
```console
$ git checkout HEAD -- python/pyarrow/_compute.pyx
$ pip install -e python --no-build-isolation
$ python -m pytest pyarrow/tests/test_compute.py -k
deserialize_rejects_non_buffers -v
collected 607 items / 603 deselected / 4 selected
pyarrow/tests/test_compute.py::test_function_options_deserialize_rejects_non_buffers[None]
PASSED [ 25%]
pyarrow/tests/test_compute.py::test_function_options_deserialize_rejects_non_buffers[1]
PASSED [ 50%]
pyarrow/tests/test_compute.py::test_function_options_deserialize_rejects_non_buffers[value2]
PASSED [ 75%]
pyarrow/tests/test_compute.py::test_function_options_deserialize_rejects_non_buffers[]
PASSED [100%]
====================== 4 passed, 603 deselected in 0.06s
=======================
```
```console
$ for v in None 1 "[]" "b''"; do python -c "
import pyarrow.compute as pc
try:
pc.FunctionOptions.deserialize($v)
except TypeError as e:
print('$v', '->', e)
"; done
None -> Argument 'buf' has incorrect type (expected pyarrow.lib.Buffer, got
NoneType)
1 -> Argument 'buf' has incorrect type (expected pyarrow.lib.Buffer, got
int)
[] -> Argument 'buf' has incorrect type (expected pyarrow.lib.Buffer, got
list)
b -> Argument 'buf' has incorrect type (expected pyarrow.lib.Buffer, got
bytes)
```
</details>
A valid buffer still round-trips, and the rest of `test_compute.py` is
unaffected.
<details>
<summary>Round-trip and full suite</summary>
```console
$ python -c "import pyarrow.compute as pc; o =
pc.ArraySortOptions(order='descending');
print(pc.FunctionOptions.deserialize(o.serialize()) == o)"
True
$ python -m pytest pyarrow/tests/test_compute.py -q
584 passed, 23 skipped, 11 warnings in 3.09s
```
</details>
### Are there any user-facing changes?
Passing a non-`Buffer` to `FunctionOptions.deserialize()` now raises
`TypeError` instead of terminating the process. Callers already passing a
`Buffer`, which is what the docstring documents, are unaffected.
* GitHub Issue: #51041
Authored-by: 1fanwang <[email protected]>
Signed-off-by: AlenkaF <[email protected]>
---
python/pyarrow/_compute.pyx | 2 +-
python/pyarrow/tests/test_compute.py | 6 ++++++
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/python/pyarrow/_compute.pyx b/python/pyarrow/_compute.pyx
index 1c98bbfdea2..da237d99fd5 100644
--- a/python/pyarrow/_compute.pyx
+++ b/python/pyarrow/_compute.pyx
@@ -630,7 +630,7 @@ cdef class FunctionOptions(_Weakrefable):
return pyarrow_wrap_buffer(c_buf)
@staticmethod
- def deserialize(buf):
+ def deserialize(Buffer buf not None):
"""
Deserialize options for a function.
diff --git a/python/pyarrow/tests/test_compute.py
b/python/pyarrow/tests/test_compute.py
index 8b2ad2b333f..6fac20a5ada 100644
--- a/python/pyarrow/tests/test_compute.py
+++ b/python/pyarrow/tests/test_compute.py
@@ -241,6 +241,12 @@ def test_option_class_equality(request):
"ArraySortOptions(order=Ascending, null_placement=AtEnd)"
[email protected]("value", [None, 1, [], b""])
+def test_function_options_deserialize_rejects_non_buffers(value):
+ with pytest.raises(TypeError):
+ pc.FunctionOptions.deserialize(value)
+
+
def test_list_functions():
assert len(pc.list_functions()) > 10
assert "add" in pc.list_functions()
