[
https://issues.apache.org/jira/browse/FLINK-40472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18115030#comment-18115030
]
Liu Liu commented on FLINK-40472:
---------------------------------
Hi [~dianfu] , I'd like to take this issue. Could you assign it to me please?
> Add Arrow-native vectorized UDF support to DataFrame API
> --------------------------------------------------------
>
> Key: FLINK-40472
> URL: https://issues.apache.org/jira/browse/FLINK-40472
> Project: Flink
> Issue Type: Sub-task
> Reporter: Liu Liu
> Priority: Major
>
> Support the following usage:
> {code:java}
> import pyarrow as pa
> import pyarrow.compute as pcfrom pyflink.dataframe import DataType, col, udf
> @udf(return_dtype=DataType.string(), batch_size=1024)
> def normalize_name(names: pa.Array) -> pa.Array:
> # Operates directly on an Arrow array without converting to pandas.
> return pc.utf8_upper(names)
> result = df.with_column(
> "normalized_name",
> normalize_name(col("name")),
> ) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
