GitHub user koertkuipers opened a pull request:
https://github.com/apache/spark/pull/16889
[SPARK-17668][SQL] Use Expressions for conversions to/from user types in
UDFs
## What changes were proposed in this pull request?
do not merge
this is a first attempt at trying to address SPARK-17688. but i do no
expect it to be sufficient.
things that bother me:
* i do not use codegen for the encoder expressions. instead i rely on
fromRow and toRow in ExpressionEncoder. that seems inefficient.
* some unnecessary wrapping in InternalRows. this is probably related to
the usage of fromRow and toRow
## How was this patch tested?
added TypedUDFSuite
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tresata/spark feat-typed-udf
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16889.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16889
commit e73c84ff75e66dee9a395c9913109a965d7d68f4
Author: Koert Kuipers
Date: 2017-02-09T20:04:36Z
got something working but not sure how good this is yet
commit bd111ae1f1a721ae2664d0e8a5810f018eb4c935
Author: Koert Kuipers
Date: 2017-02-10T01:16:34Z
pattern match to create expr1
commit 1a257c366aa1c0181dd7791fdd6eb05140c48d7e
Author: Koert Kuipers
Date: 2017-02-10T02:04:53Z
fix bug with internal row getting re-used and check results in unit tests
commit e1c337ae9e747a3dc02b939a87c9bb5c8605b86c
Author: Koert Kuipers
Date: 2017-02-10T03:26:40Z
deal with annoying style rules
commit ac09ad519437fe8efb071f354cc4387a4a95c206
Author: Koert Kuipers
Date: 2017-02-10T19:43:19Z
Merge branch 'master' into feat-typed-udf
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
