Jukka Vaaramaki created SPARK-17255:
---------------------------------------
Summary: Spark queries inside Futures occasionally fail due to
missing class definitions
Key: SPARK-17255
URL: https://issues.apache.org/jira/browse/SPARK-17255
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.0.0
Environment: OSX El Capitan, Scala 2.11.8, JDK 1.8
Reporter: Jukka Vaaramaki
Wrapping Spark queries inside Future's causes them to occasionally run in
threads that have very limited context class loaders. The class loader obtained
by calling Thread.currentThread.getContextClassLoader can be very minimal
despite any classpath definitions and missing all the Spark libraries. This
causes ClassNotFoundException or ScalaReflectionException to be thrown when
building or executing the Spark query. Sometimes an IllegalArgumentException is
thrown when none of the compression codecs are found.
An easy fix is to replace the Thread.currentThread.getContextClassLoader calls
with getClass.getClassLoader.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
