Rich-T-kid commented on code in PR #10894:
URL: https://github.com/apache/arrow-rs/pull/10894#discussion_r3975169012
##########
arrow-pyarrow/src/lib.rs:
##########
@@ -314,39 +329,79 @@ impl ToPyArrow for Schema {
}
}
-impl FromPyArrow for ArrayData {
- type_hint!(INPUT_TYPE = type_hint_identifier!("pyarrow", "Array"));
+/// Convert a Python object to [`ArrayData`] without validating the result.
+///
+/// # Safety
+///
+/// The caller must ensure the data produced by the Python object upholds the
invariants
+/// required by [`ArrayData`]. Prefer [`FromPyArrow::from_pyarrow_bound`] for
`ArrayData`, which
+/// validates the result before returning it.
+///
+/// # Example
+///
+/// ```ignore
+/// use arrow_array::{Array, Int32Array, make_array};
+/// use arrow_pyarrow::from_pyarrow_bound_unsafe;
+/// use pyo3::prelude::*;
+///
+/// /// Sums an int32 pyarrow array without paying the cost of full validation.
+/// ///
+/// /// # Safety
+/// ///
+/// /// `ob` must be a valid `pyarrow.Array` of type `int32` produced by a
trusted source.
+/// /// Passing an array with out-of-bounds offsets or mismatched buffers is
undefined behavior.
+/// #[pyfunction]
+/// unsafe fn fast_sum(ob: &Bound<'_, PyAny>) -> PyResult<i64> {
+/// // Skip full validation — we trust the source (e.g. data we produced
ourselves).
+/// // Use `ArrayData::from_pyarrow_bound` instead if the data comes from
an untrusted caller.
+/// let data = unsafe { from_pyarrow_bound_unsafe(ob)? };
+/// let array = make_array(data);
+/// let int_array = array
+/// .as_any()
+/// .downcast_ref::<Int32Array>()
+/// .ok_or_else(|| pyo3::exceptions::PyTypeError::new_err("expected
int32 array"))?;
+///
+/// Ok(int_array.iter().flatten().map(i64::from).sum())
+/// }
+/// ```
+pub unsafe fn from_pyarrow_bound_unsafe(value: &Bound<PyAny>) ->
PyResult<ArrayData> {
Review Comment:
> An option might be to introduce a new FromPyArrowUnchecked trait and
implement it on ArrayData, RecordBatch
I'll work on this and share an update.
>one issue with this is its limited to only directly calling this for one
array data; for example we cant use this for a vec of arraydata naturally, or
within a recordbatch
>one issue with this is its limited to only directly calling this for one
array data; for example we cant use this for a vec of arraydata naturally, or
within a recordbatch
are there currently use cases like this? from the two examples we've listed
above it seems to be a single invocation, not in a loop or something of that
nature
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
