timsaucer commented on code in PR #10894:
URL: https://github.com/apache/arrow-rs/pull/10894#discussion_r3967859077
##########
arrow-pyarrow/src/lib.rs:
##########
@@ -314,39 +329,79 @@ impl ToPyArrow for Schema {
}
}
-impl FromPyArrow for ArrayData {
- type_hint!(INPUT_TYPE = type_hint_identifier!("pyarrow", "Array"));
+/// Convert a Python object to [`ArrayData`] without validating the result.
+///
+/// # Safety
+///
+/// The caller must ensure the data produced by the Python object upholds the
invariants
+/// required by [`ArrayData`]. Prefer [`FromPyArrow::from_pyarrow_bound`] for
`ArrayData`, which
+/// validates the result before returning it.
+///
+/// # Example
+///
+/// ```ignore
+/// use arrow_array::{Array, Int32Array, make_array};
+/// use arrow_pyarrow::from_pyarrow_bound_unsafe;
+/// use pyo3::prelude::*;
+///
+/// /// Sums an int32 pyarrow array without paying the cost of full validation.
+/// ///
+/// /// # Safety
+/// ///
+/// /// `ob` must be a valid `pyarrow.Array` of type `int32` produced by a
trusted source.
+/// /// Passing an array with out-of-bounds offsets or mismatched buffers is
undefined behavior.
+/// #[pyfunction]
+/// unsafe fn fast_sum(ob: &Bound<'_, PyAny>) -> PyResult<i64> {
+/// // Skip full validation — we trust the source (e.g. data we produced
ourselves).
+/// // Use `ArrayData::from_pyarrow_bound` instead if the data comes from
an untrusted caller.
+/// let data = unsafe { from_pyarrow_bound_unsafe(ob)? };
+/// let array = make_array(data);
+/// let int_array = array
+/// .as_any()
+/// .downcast_ref::<Int32Array>()
+/// .ok_or_else(|| pyo3::exceptions::PyTypeError::new_err("expected
int32 array"))?;
+///
+/// Ok(int_array.iter().flatten().map(i64::from).sum())
+/// }
+/// ```
+pub unsafe fn from_pyarrow_bound_unsafe(value: &Bound<PyAny>) ->
PyResult<ArrayData> {
Review Comment:
I know it increases the scope of this PR, but @Tpt 's suggestion of a new
trait is IMO the cleanest final solution. That way if we know for sure we don't
need checking we can go strait to that version and if we need safety we can
keep the existing method.
I'll try to think about how to do some performance testing on this and what
the impacts might be for things like datafusion-python where we use these
interfaces.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
