GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/21611
[SPARK-24569][SQL] Aggregator with output type Option should produce
consistent schema
## What changes were proposed in this pull request?
SQL `Aggregator` with output type `Option[Boolean]` creates column of type
`StructType`. It's not in consistency with a Dataset of similar java class.
This changes the way `definedByConstructorParams` checks given type. For
`Option[_]`, it goes to check its type argument.
## How was this patch tested?
Added test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 SPARK-24569
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21611.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21611
----
commit dd4ea61ac1c2beaf8ee897b1533e2088c6f8364a
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-06-22T03:44:33Z
Aggregator with output type Option[Boolean] should produce consistent
schema.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
