Github user jliwork commented on the pull request: https://github.com/apache/spark/pull/9247#issuecomment-150992372 @cloud-fan Thank you very much for your comment! Sorting on array of NULL type does not make sense to me, either. That's why my fix is to block this kind of sorting for sort_array. 1)I have checked Hive's behavior. Hive does not throws any exception when sorting an array of NULLs. It will simple return the array of NULLs back. With my fix spark's sort_array will return the exact same result as Hive. 2) Regarding sorting on struct, I could not find any existing UDF like sort_struct. Hive does not seems to offer such UDF, either. I'm not very familiar with dataframe's struct type as I'm new to Spark and still learning. Please let me know if you have any other questions or concerns. Thank again.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org