GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/17639
[SPARK-19716][SQL][follow-up] UnresolvedMapObjects should always be
serializable
## What changes were proposed in this pull request?
In https://github.com/apache/spark/pull/17398 we introduced
`UnresolvedMapObjects` as a placeholder of `MapObjects`. Unfortunately
`UnresolvedMapObjects` is not serializable as its `function` may reference
Scala `Type` which is not serializable.
Ideally this is fine, as we will never serialize and send unresolved
expressions to executors. However users may accidentally do this, e.g.
mistakenly reference an encoder instance when implementing `Aggregator`, we
should fix it so that it's just a performance issue(more network traffic) and
should not fail the query.
## How was this patch tested?
N/A
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark minor
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17639.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17639
----
commit bb0a14a391e7316a1d5caee900f295c9487c6e8a
Author: Wenchen Fan <[email protected]>
Date: 2017-04-14T13:15:36Z
UnresolvedMapObjects should always be serializable
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
