[
https://issues.apache.org/jira/browse/SPARK-13594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reynold Xin updated SPARK-13594:
--------------------------------
Description:
Once we implement Dataset-equivalent API in Python, we'd need to change the
return type of map, flatMap, and mapPartitions. In this case, we should just
remove them from Python DataFrame now in 2.0, so we don't break APIs in 2.x.
Users can still use those after the removal, but must prefix "rdd" to it. For
example, df.rdd.map, df.rdd.flatMap, and df.rdd.mapPartitions.
was:
Once we implement Dataset-equivalent API in Python, we'd need to change the
return type of map, flatMap, and mapPartitions. In this case, we should just
remove them from Python DataFrame now in 2.0, so we don't break APIs in 2.x.
> remove typed operations (map, flatMap, mapPartitions) from Python DataFrame
> ----------------------------------------------------------------------------
>
> Key: SPARK-13594
> URL: https://issues.apache.org/jira/browse/SPARK-13594
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Wenchen Fan
> Assignee: Wenchen Fan
> Fix For: 2.0.0
>
>
> Once we implement Dataset-equivalent API in Python, we'd need to change the
> return type of map, flatMap, and mapPartitions. In this case, we should just
> remove them from Python DataFrame now in 2.0, so we don't break APIs in 2.x.
> Users can still use those after the removal, but must prefix "rdd" to it. For
> example, df.rdd.map, df.rdd.flatMap, and df.rdd.mapPartitions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
