Yikun commented on pull request #32276: URL: https://github.com/apache/spark/pull/32276#issuecomment-824472080
> As you already noticed these methods are private and, as such, not intended for end-user. @zero323 Maybe I bring some misunderstanding in previous , I didn't expose the **private withColumns** API in PySpark, just to match the ability of the scala withColumn API. That means, the scala [withColumn API](https://github.com/apache/spark/blob/b5241c97b17a1139a4ff719bfce7f68aef094d95/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L2396) can receive multiple columns now, and support by calling the [internal withColumns API](https://github.com/apache/spark/blob/b5241c97b17a1139a4ff719bfce7f68aef094d95/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L2396-L2402). > we should think about having an API in Scala too to match. It was sort of rejected once at SPARK-12225 @HyukjinKwon See the first reply for misunderstanding, actually, SPARK-12225 have already support the multiple columns adding support on scala withColumn API in https://github.com/apache/spark/commit/3ca367083e196e6487207211e6c49d4bbfe31288 ([SPARK-22001](https://issues.apache.org/jira/browse/SPARK-22001)) > Additionally, naming would be rather confusing. > we should discuss adding such functionality for all APIs and having less confusing name. Agree, the scala API withColumn receive some confusing naming input to support the muliple columns adding,I will send an email to dev ML to decribe these confusing naming things. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
