GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/14906
Disable kryo in thriftserver
In SPARK-4761 / #3621 (December 2014) we enabled Kryo serialization by
default in the Spark Thrift Server. However, I don't think that the original
rationale for doing this still holds now that most Spark SQL serialization is
now performed via encoders and our UnsafeRow format.
In addition, the use of Kryo as the default serializer can introduce
performance problems because the creation of new KryoSerializer instances is
expensive and we haven't performed instance-reuse optimizations in several code
paths (including DirectTaskResult deserialization).
Given all of this, I propose to revert back to using JavaSerializer as the
default serializer in the Thrift Server.
/cc @liancheng
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark disable-kryo-in-thriftserver
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14906.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14906
----
commit aa18bb69f4ec60afffe2e5dc3c3bc6ac860b7821
Author: Josh Rosen <[email protected]>
Date: 2016-09-01T00:03:47Z
Disable use of KryoSerializer in Thrift Server
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
