[
https://issues.apache.org/jira/browse/SPARK-12225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15048432#comment-15048432
]
Sun Rui commented on SPARK-12225:
---------------------------------
[[email protected]]
This comes from supporting mutate() function in SparkR. mutate() is a function
in the popular data manipulation dplyr package, which allows adding and
replacing multiple columns in a call.
The implementation can't be as simple as calling withColumn() multiple times,
as each call to withColumn() returns a new DataFrame, while the remaining
columns to be added or for replacement are Columns referring to the old
DataFrame. So the implementation is not trivial, and you can refer to
https://github.com/apache/spark/pull/10220.
It would simplify the SparkR's implementation if there is a multiple-column
version of withColumn.
Another motivation for this JIRA is that we just added drop() for multiple
columns. So from API parity's point of view, it would be good to have a
multiple-column version of withColumn.
> Support adding or replacing multiple columns at once in DataFrame API
> ---------------------------------------------------------------------
>
> Key: SPARK-12225
> URL: https://issues.apache.org/jira/browse/SPARK-12225
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Affects Versions: 1.5.2
> Reporter: Sun Rui
>
> Currently, withColumn() method of DataFrame supports adding or replacing only
> single column. It would be convenient to support adding or replacing multiple
> columns at once.
> Also withColumnRenamed() supports renaming only single column.It would also
> be convenient to support renaming multiple columns at once.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
