panbingkun commented on PR #41984: URL: https://github.com/apache/spark/pull/41984#issuecomment-1634069791
No further others similar case have been found so far, Also, I would like to use this place to discuss some question: - In [FunctionRegistry](https://github.com/apache/spark/blob/8bb07388ea664303d0d22b03cca11a46498b772d/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala#L300), the classification of some functions is incorrect, is it worth doing? eg: https://github.com/apache/spark/blob/8bb07388ea664303d0d22b03cca11a46498b772d/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala#L357-L363 Obviously, the function `Abs` belongs to https://github.com/apache/spark/blob/8bb07388ea664303d0d22b03cca11a46498b772d/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala#L384-L388 - Do we need to align the classification of functions in scala with that in Python? eg: `EqualNull` In scala, it belongs to `Misc Function` https://github.com/apache/spark/blob/8bb07388ea664303d0d22b03cca11a46498b772d/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala#L1114-L1117 https://github.com/apache/spark/blob/8bb07388ea664303d0d22b03cca11a46498b772d/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala#L721-L748 In python, it belongs to `Predicate Functions` https://github.com/apache/spark/blob/8bb07388ea664303d0d22b03cca11a46498b772d/python/docs/source/reference/pyspark.sql/functions.rst?plain=1#L514-L520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
