Josh Rosen created SPARK-13926:
----------------------------------
Summary: Automatically use Kryo serializer when it is known to be
safe
Key: SPARK-13926
URL: https://issues.apache.org/jira/browse/SPARK-13926
Project: Spark
Issue Type: Improvement
Components: Shuffle
Reporter: Josh Rosen
Assignee: Josh Rosen
Because ClassTags are available when constructing ShuffledRDD we can use them
to automatically use Kryo for shuffle serialization when the RDD's types are
guaranteed to be compatible with Kryo (e.g. RDDs whose key, value, and/or
combiner types are primitives, arrays of primitives, or strings). This is
likely to result in a large performance gain for many RDD API workloads.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
