GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/22749
[WIP][SPARK-25746][SQL] Refactoring ExpressionEncoder to get rid of flat
flag
## What changes were proposed in this pull request?
This is inspired during implementing #21732. For now `ScalaReflection`
needs to consider how `ExpressionEncoder` uses generated serializers and
deserializers. And `ExpressionEncoder` has a weird `flat` flag. After
discussion with @cloud-fan, it seems to be better to refactor
`ExpressionEncoder`. It should make SPARK-24762 easier to do.
## How was this patch tested?
Existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 SPARK-24762-refactor
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22749.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22749
----
commit e1b5deebe715479125c8878f0c90a55dc9ab3e85
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-07-09T03:42:04Z
Aggregator should be able to use Option of Product encoder.
commit 80506f4e98184ccd66dbaac14ec52d69c358020d
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-07-13T04:40:55Z
Enable top-level Option of Product encoders.
commit ed3d5cb697b10af2e2cf4c78ab521d4d0b2f3c9b
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-08-24T04:26:28Z
Remove topLevel parameter.
commit 9fc3f6165156051142a8366a32726badaaa16bb7
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-08-24T04:37:39Z
Merge remote-tracking branch 'upstream/master' into SPARK-24762
commit 5f95bd0cf1bd308c7df55c41caef7a9f19368f5d
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-08-24T04:42:33Z
Remove useless change.
commit a4f04055b2ba22f371663565710328791942855a
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-08-24T14:38:16Z
Add more tests.
commit c1f798f7e9cba0d04223eed06f1b1f547ec29dc5
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-08-25T01:52:01Z
Add test.
commit 80e11d289d7775863cb9c28b2c1d4364292048a4
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-10-06T04:06:57Z
Merge remote-tracking branch 'upstream/master' into SPARK-24762
commit 0f029b0a28700334dc6334f1ad89b3124f235a51
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-10-06T04:40:07Z
Improve code comments.
commit d755e8406f06117ccc96b8f19debab6b2a736e10
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-10-15T09:55:03Z
Refactoring ExpressionEncoder.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
