GitHub user techaddict opened a pull request:
https://github.com/apache/spark/pull/22031
[TODO][SPARK-23932][SQL] Higher order function zip_with
## What changes were proposed in this pull request?
Merges the two given arrays, element-wise, into a single array using
function. If one array is shorter, nulls are appended at the end to match the
length of the longer array, before applying function:
```
SELECT zip_with(ARRAY[1, 3, 5], ARRAY['a', 'b', 'c'], (x, y) -> (y,
x)); -- [ROW('a', 1), ROW('b', 3), ROW('c', 5)]
SELECT zip_with(ARRAY[1, 2], ARRAY[3, 4], (x, y) -> x + y); -- [4, 6]
SELECT zip_with(ARRAY['a', 'b', 'c'], ARRAY['d', 'e', 'f'], (x, y) ->
concat(x, y)); -- ['ad', 'be', 'cf']
SELECT zip_with(ARRAY['a'], ARRAY['d', null, 'f'], (x, y) ->
coalesce(x, y)); -- ['a', null, 'f']
```
## How was this patch tested?
Added tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/techaddict/spark SPARK-23932
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22031.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22031
----
commit 03d19cee425be90a61b60163ff9d6740716d45a6
Author: Sandeep Singh <sandeep@...>
Date: 2018-08-03T04:15:00Z
.
commit 6f91777de93121d668ff11e7701f449bb4c96337
Author: Sandeep Singh <sandeep@...>
Date: 2018-08-04T22:00:38Z
fix description
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
