GitHub user jliwork opened a pull request:
https://github.com/apache/spark/pull/9247
[SPARK-11277] [SQL] sort_array throws exception scala.MatchError
I'm new to spark. I was trying out the sort_array function then hit this
exception. I looked into the spark source code. I found the root cause is that
sort_array does not check for an array of NULLs. It's not meaningful to sort an
array of entirely NULLs anyway.
I'm adding a check on the input array type to SortArray. If the array
consists of NULLs entirely, there is no need to sort such array. I have also
added a test case for this.
Please help to review my fix. Thanks!
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jliwork/spark SPARK-11277
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/9247.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #9247
----
commit 998b8b84f2667b1d16c732d63ba70ab6bfc5b56e
Author: Jia Li <[email protected]>
Date: 2015-10-23T09:06:26Z
[SPARK-11277] [SQL] sort_array throws exception scala.MatchError
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
