Liu Liu created FLINK-40433:
-------------------------------
Summary: Add map and map_batches support to DataFrame API
Key: FLINK-40433
URL: https://issues.apache.org/jira/browse/FLINK-40433
Project: Flink
Issue Type: Sub-task
Components: API / Python
Reporter: Liu Liu
Add Python-native row and batch transformations:
{code:java}
def map(
self,
func,
*,
return_dtype=None,
concurrency=None,
) -> DataFrame {code}
{{map}} receives each row as {{Dict[str, Any]}} and returns a dictionary
representing the output row.
{code:java}
def map_batches(
self,
func,
*,
return_dtype=None,
batch_format,
concurrency=None,
batch_size=None,
) -> DataFrame {code}
{{map_batches}} should support:
* {{dict[str, pandas.Series]}} input/output in pandas mode.
* {{dict[str, pyarrow.Array]}} input/output in Arrow mode.
{{}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
