Re: Building Spark with a Custom Version of Hadoop: HDFS ClassNotFoundException

2016-02-11 Thread Ted Yu
Hdfs class is in hadoop-hdfs-XX.jar Can you check the classpath to see if the above jar is there ? Please describe the command lines you used for building hadoop / Spark. Cheers On Thu, Feb 11, 2016 at 5:15 PM, Charlie Wright wrote: > I am having issues trying to run a test job on a built ver

Building Spark with a Custom Version of Hadoop: HDFS ClassNotFoundException

2016-02-11 Thread Charlie Wright
I am having issues trying to run a test job on a built version of Spark with a custom Hadoop JAR. My custom hadoop version runs without issues and I can run jobs from a precompiled version of Spark (with Hadoop) no problem. However, whenever I try to run the same Spark example on the Spark versi