[
https://issues.apache.org/jira/browse/FLINK-40433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Liu updated FLINK-40433:
----------------------------
Description:
Add Python-native row and batch transformations:
{code:java}
def map(
self,
func,
*,
return_dtype=None,
concurrency=None,
) -> DataFrame {code}
{{map}} receives each row as {{Dict[str, Any]}} and returns a dictionary
representing the output row.
{code:java}
def map_batches(
self,
func,
*,
return_dtype=None,
batch_format,
concurrency=None,
batch_size=None,
) -> DataFrame {code}
{{map_batches}} should support:
* {{dict[str, pandas.Series]}} input/output in pandas mode.
* {{dict[str, pyarrow.Array]}} input/output in Arrow mode.
was:
Add Python-native row and batch transformations:
{code:java}
def map(
self,
func,
*,
return_dtype=None,
concurrency=None,
) -> DataFrame {code}
{{map}} receives each row as {{Dict[str, Any]}} and returns a dictionary
representing the output row.
{code:java}
def map_batches(
self,
func,
*,
return_dtype=None,
batch_format,
concurrency=None,
batch_size=None,
) -> DataFrame {code}
{{map_batches}} should support:
* {{dict[str, pandas.Series]}} input/output in pandas mode.
* {{dict[str, pyarrow.Array]}} input/output in Arrow mode.
{{}}
> Add map and map_batches support to DataFrame API
> ------------------------------------------------
>
> Key: FLINK-40433
> URL: https://issues.apache.org/jira/browse/FLINK-40433
> Project: Flink
> Issue Type: Sub-task
> Components: API / Python
> Reporter: Liu Liu
> Priority: Major
>
> Add Python-native row and batch transformations:
> {code:java}
> def map(
> self,
> func,
> *,
> return_dtype=None,
> concurrency=None,
> ) -> DataFrame {code}
> {{map}} receives each row as {{Dict[str, Any]}} and returns a dictionary
> representing the output row.
> {code:java}
> def map_batches(
> self,
> func,
> *,
> return_dtype=None,
> batch_format,
> concurrency=None,
> batch_size=None,
> ) -> DataFrame {code}
> {{map_batches}} should support:
> * {{dict[str, pandas.Series]}} input/output in pandas mode.
> * {{dict[str, pyarrow.Array]}} input/output in Arrow mode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
