Hello users,
While using Spark to parse my files which is in the format of gz and, each
of which can be in the size of 150MB (~2GB. when unzipped). But, I read
the input as gz files as such. While parsing, I get this error
Exception in thread "delete Spark temp dir
/tmp/spark-daa280f8-5024-431e-a7f8-b7e77b3a876e" Exception in thread
"delete Spark temp dir /tmp/spark-eca1eeb9-6a9a-4528-9ef4-2ee78ed3a634"
java.lang.NoClassDefFoundError: org/apache/spark/util/Utils$$anonfun$1
at
org.apache.spark.util.Utils$.hasRootAsShutdownDeleteDir(Utils.scala:162)
at org.apache.spark.util.Utils$$anon$4.run(Utils.scala:197)
java.lang.NoClassDefFoundError: org/apache/spark/util/Utils$$anonfun$1
at
org.apache.spark.util.Utils$.hasRootAsShutdownDeleteDir(Utils.scala:162)
at org.apache.spark.util.Utils$$anon$4.run(Utils.scala:197)
Caused by: java.lang.ClassNotFoundException:
org.apache.spark.util.Utils$$anonfun$1
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
... 2 more
Exception in thread "delete Spark temp dir
/tmp/spark-45453f82-6b77-4df8-81c1-ed45a9d59a1d"
java.lang.NoClassDefFoundError: org/apache/spark/util/Utils$$anonfun$1
at
org.apache.spark.util.Utils$.hasRootAsShutdownDeleteDir(Utils.scala:162)
at org.apache.spark.util.Utils$$anon$4.run(Utils.scala:197)
Exception in thread "delete Spark local dirs"
java.lang.NoClassDefFoundError:
org/apache/spark/storage/DiskStore$$anon$1$$anonfun$run$1
at
org.apache.spark.storage.DiskStore$$anon$1.run(DiskStore.scala:300)
Caused by: java.lang.ClassNotFoundException:
org.apache.spark.storage.DiskStore$$anon$1$$anonfun$run$1
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
Any ideas on this would be appreciated. Thanks!
Regards,
Ram
