[
https://issues.apache.org/jira/browse/SPARK-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14341819#comment-14341819
]
Sean Owen commented on SPARK-6069:
----------------------------------
No I set it kind of preemptively. I don't know that I serialize any Guava
classes though, come to think of it.
I am using YARN + Hadoop 2.5.
I don't think it should be necessary in general. Guava is a strange special
case, so though it worth trying.
If you have the energy, you might try 1.3.0-SNAPSHOT since I see a few things
fixed that may be relevant:
https://issues.apache.org/jira/browse/SPARK-4877
https://issues.apache.org/jira/browse/SPARK-4660
> Deserialization Error ClassNotFoundException with Kryo, Guava 14
> ----------------------------------------------------------------
>
> Key: SPARK-6069
> URL: https://issues.apache.org/jira/browse/SPARK-6069
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.2.1
> Environment: Standalone one worker cluster on localhost, or any
> cluster
> Reporter: Pat Ferrel
> Priority: Critical
>
> A class is contained in the jars passed in when creating a context. It is
> registered with kryo. The class (Guava HashBiMap) is created correctly from
> an RDD and broadcast but the deserialization fails with ClassNotFound.
> The work around is to hard code the path to the jar and make it available on
> all workers. Hard code because we are creating a library so there is no easy
> way to pass in to the app something like:
> spark.executor.extraClassPath /path/to/some.jar
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
