GitHub user ueshin opened a pull request:
https://github.com/apache/spark/pull/21954
[SPARK-23908][SQL] Add transform function.
## What changes were proposed in this pull request?
This pr adds `transform` function which transforms elements in an array
using the function.
Optionally we can take the index of each element as the second argument.
```sql
> SELECT transform(array(1, 2, 3), x -> x + 1);
array(2, 3, 4)
> SELECT transform(array(1, 2, 3), (x, i) -> x + i);
array(1, 3, 5)
```
## How was this patch tested?
Added tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ueshin/apache-spark
issues/SPARK-23908/transform
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21954.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21954
----
commit e860249c96854daefccc580ca52e3fad7d1acf67
Author: Takuya UESHIN <ueshin@...>
Date: 2018-08-01T03:46:00Z
Add `LambdaFunction` and its parser.
commit 06825b6a3ae6b3085c4b4e5c010dffa75e988801
Author: Takuya UESHIN <ueshin@...>
Date: 2018-08-01T05:52:40Z
Add `ResolveHigherOrderFunctions`.
commit 17ab2ffc73a664ba2b00d49f0835faff055274b0
Author: Takuya UESHIN <ueshin@...>
Date: 2018-08-01T05:59:32Z
Add `ArrayTransform`.
commit 95a06b4fe3f660c617a6a53bf0cccc5a0a62306b
Author: Takuya UESHIN <ueshin@...>
Date: 2018-08-01T08:06:01Z
Test in sql/core.
commit 4448d0b7a085c1613dc0dd52009b6d50388ec605
Author: Takuya UESHIN <ueshin@...>
Date: 2018-08-01T09:20:14Z
Add negative cases.
commit abc685f86ee205f2a64b065c98a65fc2d36bfd75
Author: Takuya UESHIN <ueshin@...>
Date: 2018-08-01T09:46:00Z
Add sql file.
commit ee450c5ef3f99d3bbf8dbbd05273bc63005bbccb
Author: Takuya UESHIN <ueshin@...>
Date: 2018-08-01T10:15:44Z
Replace lambda variable in function by one in arguments to make sure the
variables are the same as them in arguments.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
