GitHub user ianoc opened a pull request:
https://github.com/apache/spark/pull/1377
[SQL][CORE] SPARK-2102
Adds in an option into the kryo serializer to enable requiring registration
Adds a resource pool in the SparkSqlSerializer code paths to speed up
serialization of generic types.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ianoc/spark feature/SPARK-2102
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1377.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1377
----
commit 665805a49f44ec5094a05589721fb2c040cfc748
Author: Ian O Connell <[email protected]>
Date: 2014-07-11T21:41:02Z
Add a spark.kryo.registrationRequired option for configuring the Kryo
Serializer
commit 4e5c34294edcf57bf219790c97a3c240115531c3
Author: Ian O Connell <[email protected]>
Date: 2014-07-11T21:42:16Z
Register the SparkConf for kryo, it gets swept into serialization
commit f3735c88c551d2e6ccab6d26e8a6ef2629d35644
Author: Ian O Connell <[email protected]>
Date: 2014-07-11T21:44:28Z
Add using a kryo resource pool for the SqlSerializer
commit f92c294ffa61c453d8ff8166694a9c6afc7c990a
Author: Ian O Connell <[email protected]>
Date: 2014-07-11T22:29:15Z
Add docs for new KryoSerializer option
commit 20e855574c83fbe0efe109fa34df0d2410ad6e25
Author: Ian O Connell <[email protected]>
Date: 2014-07-11T22:29:22Z
Slight style change
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
