GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/21732
[SPARK-24762][SQL] Aggregator should be able to use Option of Product
encoder
## What changes were proposed in this pull request?
Encoders has a limitation that we can't construct encoders for Option of
Product at top-level, because in SparkSQL entire row can't be null.
However for some use cases such as Aggregator, it should be able to
construct encoders for Option of Product at non top-level.
## How was this patch tested?
Added test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 SPARK-24762
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21732.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21732
----
commit e1b5deebe715479125c8878f0c90a55dc9ab3e85
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-07-09T03:42:04Z
Aggregator should be able to use Option of Product encoder.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
