[
https://issues.apache.org/jira/browse/SPARK-11461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-11461.
-------------------------------
Resolution: Not A Problem
> ObjectFile saving/loading should use configured serializer
> ----------------------------------------------------------
>
> Key: SPARK-11461
> URL: https://issues.apache.org/jira/browse/SPARK-11461
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 1.5.1
> Reporter: Ryan Williams
>
> [RDD.saveAsObjectFile|https://github.com/apache/spark/blob/v1.5.1/core/src/main/scala/org/apache/spark/rdd/RDD.scala#L1452]
> and
> [SparkContext.objectFile|https://github.com/apache/spark/blob/v1.5.1/core/src/main/scala/org/apache/spark/SparkContext.scala#L1223]
> use
> [Utils.serialize|https://github.com/apache/spark/blob/v1.5.1/core/src/main/scala/org/apache/spark/util/Utils.scala#L78-L85]
> and
> [Utils.deserialize|https://github.com/apache/spark/blob/v1.5.1/core/src/main/scala/org/apache/spark/util/Utils.scala#L94-L105]
> which are hard-coded to use Java SerDe rather than the serializer configured
> via the {{spark.serializer}} conf param.
> I'd like to write RDDs as Object-/Sequence-Files using e.g. Kryo serde
> instead of Java; is there a way to do this, or any reason that Spark
> currently only supports Java?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
