Hi folks, running into a pretty strange issue: I'm setting spark.executor.extraClassPath spark.driver.extraClassPath
to point to some external JARs. If I set them in spark-defaults.conf everything works perfectly. However, if I remove spark-defaults.conf and just create a SparkConf and call .set("spark.executor.extraClassPath","...) .set("spark.driver.extraClassPath",...) I get ClassNotFound exceptions from Hadoop Conf: Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.ceph.CephFileSystem not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1493) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1585) This seems like a bug to me -- or does spark-defaults.conf somehow get processed differently? I have dumped out sparkConf.toDebugString and in both cases (spark-defaults.conf/in code sets) it seems to have the same values in it...