HyukjinKwon commented on issue #23879: [SPARK-26979][SQL] Add missing column name support for SQL functions URL: https://github.com/apache/spark/pull/23879#issuecomment-466761648 Of course it deprecates and not just removes. This itself is a trivial change but for consistency we shouldn't make exception unless there's a special reason for that. Spark's now going ahead major version bump up to Spark 3. We should fix something we have postponed in lower versions. PySpark has the same problem as well. I wouldn't fix it what's already there from Spark 1.x - if it was a new API, I would have been okay since we already have the inconsistency there. We actually fix it in Scala side and see what's going to happen. If Spark's going to remove string arguments, it should also be considered to remove string as columns in PySpark APIs as well.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
