asmello commented on issue #23882: [SPARK-26979][PYTHON] Add missing string column name support for some SQL functions URL: https://github.com/apache/spark/pull/23882#issuecomment-471104496 @srowen sorry, I had missed the email, but I think @rxin means about situations like `array_contains("foo", "bar")` where it's ambiguous if "bar" represents a column name or a literal value. Since #21581 the Scala API accepts either a literal or a Column object, but not a column name, which is indeed confusing. PySpark doesn't expose that functionality and only ever takes a literal where one is allowed, so there's no risk of ambiguity - but that does make for another kind of consistency break. You might expect `array_contains(col("foo"), col("bar"))` to work, but it doesn't. Might be worth patching this together with `.drop()` and the stats functions, but, again, it's a different kind of problem.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
