Github user jliwork commented on the pull request:
https://github.com/apache/spark/pull/9247#issuecomment-150992372
@cloud-fan Thank you very much for your comment! Sorting on array of NULL
type does not make sense to me, either. That's why my fix is to block this kind
of sorting for sort_array.
1)I have checked Hive's behavior. Hive does not throws any exception when
sorting an array of NULLs. It will simple return the array of NULLs back. With
my fix spark's sort_array will return the exact same result as Hive.
2) Regarding sorting on struct, I could not find any existing UDF like
sort_struct. Hive does not seems to offer such UDF, either. I'm not very
familiar with dataframe's struct type as I'm new to Spark and still learning.
Please let me know if you have any other questions or concerns. Thank
again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
