[
https://issues.apache.org/jira/browse/SPARK-4354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14219418#comment-14219418
]
hoelog commented on SPARK-4354:
-------------------------------
Changing codec to LZ4 will also help.
I also tested it.
http://apache-spark-user-list.1001560.n3.nabble.com/How-to-avoid-use-snappy-compression-when-saveAsSequenceFile-td17350.html
{panel:title=conf/spark-defaults.conf}
spark.io.compression.codec org.apache.spark.io.LZ4CompressionCodec
{panel}
> 14/11/12 09:39:00 WARN TaskSetManager: Lost task 5.0 in stage 0.0 (TID 5,
> HYD-RNDNW-VFRCO-RCORE2): java.lang.NoClassDefFoundError: Could not initialize
> class org.xerial.snappy.Snappy
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-4354
> URL: https://issues.apache.org/jira/browse/SPARK-4354
> Project: Spark
> Issue Type: Question
> Components: Examples
> Affects Versions: 1.1.0
> Environment: Linux
> Reporter: Shyam
> Labels: newbie
> Attachments: client-exception.txt
>
>
> Prebuilt Spark for Hadoop 2.4 installed in 4 redhat linux machines
> Standalone cluster mode.
> Machine 1(Master)
> Machine 2(Worker node 1)
> Machine 3(Worker node 2)
> Machine 4(Client for executing spark examples)
> I ran below mentioned command in Machine 4 then got exception mentioned in
> the summary of this issue.
> sh spark-submit --class org.apache.spark.examples.SparkPi --jars
> /FS/lib/spark-assembly-1.1.0-hadoop2.4.0.jar --master spark://MasterIP:7077
> --deploy-mode client /FS/lib/spark-examples-1.1.0-hadoop2.4.0.jar 10
> java.lang.NoClassDefFoundError: Could not initialize class
> org.xerial.snappy.Snappy
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
