[
https://issues.apache.org/jira/browse/ARROW-17827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joris Van den Bossche updated ARROW-17827:
------------------------------------------
Labels: expressions query-engine (was: expressions query)
> [Python] Allow calling UDF kernels with field/scalar expressions
> ----------------------------------------------------------------
>
> Key: ARROW-17827
> URL: https://issues.apache.org/jira/browse/ARROW-17827
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Joris Van den Bossche
> Priority: Major
> Labels: expressions, query-engine
>
> From https://github.com/apache/arrow/pull/13687#issuecomment-1240399112,
> where it came up while adding documentation on how to use UDFs in Python.
> When just wanting to invoke a UDF with arrays, you can do
> {{pc.call_function("my_udf", [pc.field("a")])}}.
> But if you want to use your UDF in a context that needs an expression (eg a
> dataset projection), you need to be able to call the UDF with expressions as
> argument. And currently, the {{pc.call_function}} doesn't work that way (it
> expects actual, materialized arrays/scalars as arguments). As a workaround,
> you can use the private {{Expression._call}}:
> {code:python}
> # doesn't work with expressions
> >>> pc.call_function("my_udf", [pc.field("col")])
> ...
> TypeError: Got unexpected argument type <class 'pyarrow._compute.Expression'>
> for compute function
> # workaround
> >>> pc.Expression._call("my_udf", [pc.field("col")])
> <pyarrow.compute.Expression my_udf(col)>
> {code}
> So we should try to improve the usability here. Some options:
> * See if we can change {{pc.call_function}} to also accept Expressions as
> arguments
> * Make the {{_call}} public, so one can do {{pc.Expression.call("my_udf",
> [..])}}
> cc [~westonpace] [~vibhatha]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
