Hi, Did you install CDH 4.2.0 MR2 (yarn) ? If then, you should type, "SPARK_HADOOP_VERSION=2.0.0-cdh4.2.0 SPARK_YARN=true sbt/sbt assembly” Try it please.
On Nov 22, 2013, at 7:13 AM, [email protected] wrote: > > Any advice what could be the issue here? > > My SPARK_HADOOP_VERSION=2.0.0-cdh4.2.0 > > [root@kserv06 spark-0.8.0-incubating]# sbt/sbt assembly > [info] Loading project definition from > /home/spark/spark-0.8.0-incubating/project/project > [info] Loading project definition from > /home/spark/spark-0.8.0-incubating/project > [info] Set current project to root (in build > file:/home/spark/spark-0.8.0-incubating/) > [info] Compiling 258 Scala sources and 16 Java sources to > /home/spark/spark-0.8.0-incubating/core/target/scala-2.9.3/classes... > [warn] > /home/spark/spark-0.8.0-incubating/core/src/main/scala/org/apache/hadoop/mapred/SparkHadoopMapRedUtil.scala:34: > constructor TaskAttemptID in class TaskAttemptID is deprecated: see > corresponding Javadoc for more information. > [warn] new TaskAttemptID(jtIdentifier, jobId, isMap, taskId, attemptId) > [warn] ^ > [warn] > /home/spark/spark-0.8.0-incubating/core/src/main/scala/org/apache/spark/SparkContext.scala:401: > constructor Job in class Job is deprecated: see corresponding Javadoc for > more information. > [warn] val job = new NewHadoopJob(conf) > [warn] ^ > [warn] > /home/spark/spark-0.8.0-incubating/core/src/main/scala/org/apache/spark/SparkHadoopWriter.scala:129: > method cleanupJob in class OutputCommitter is deprecated: see corresponding > Javadoc for more information. > [warn] getOutputCommitter().cleanupJob(getJobContext()) > [warn] ^ > [warn] > /home/spark/spark-0.8.0-incubating/core/src/main/scala/org/apache/spark/SparkHadoopWriter.scala:170: > constructor TaskID in class TaskID is deprecated: see corresponding Javadoc > for more information. > [warn] new TaskAttemptID(new TaskID(jID.value, true, splitID), > attemptID)) > [warn] ^ > [warn] > /home/spark/spark-0.8.0-incubating/core/src/main/scala/org/apache/spark/SparkHadoopWriter.scala:198: > method makeQualified in class Path is deprecated: see corresponding Javadoc > for more information. > [warn] outputPath = outputPath.makeQualified(fs) > [warn] ^ > [warn] > /home/spark/spark-0.8.0-incubating/core/src/main/scala/org/apache/spark/rdd/CheckpointRDD.scala:102: > method getDefaultReplication in class FileSystem is deprecated: see > corresponding Javadoc for more information. > [warn] fs.create(tempOutputPath, false, bufferSize, > fs.getDefaultReplication, blockSize) > [warn] ^ > [warn] > /home/spark/spark-0.8.0-incubating/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala:554: > constructor Job in class Job is deprecated: see corresponding Javadoc for > more information. > [warn] val job = new NewAPIHadoopJob(conf) > [warn] ^ > [warn] > /home/spark/spark-0.8.0-incubating/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala:592: > method cleanupJob in class OutputCommitter is deprecated: see corresponding > Javadoc for more information. > [warn] jobCommitter.cleanupJob(jobTaskContext) > [warn] ^ > [warn] > /home/spark/spark-0.8.0-incubating/core/src/main/scala/org/apache/spark/scheduler/InputFormatInfo.scala:98: > constructor Job in class Job is deprecated: see corresponding Javadoc for > more information. > [warn] val job = new Job(conf) > [warn] ^ > [warn] 9 warnings found > [error] ---------- > [error] 1. WARNING in > /home/spark/spark-0.8.0-incubating/core/src/main/java/org/apache/spark/network/netty/FileClient.java > (at line 22) > [error] import io.netty.channel.ChannelFuture; > [error] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > [error] The import io.netty.channel.ChannelFuture is never used > [error] ---------- > [error] 2. WARNING in > /home/spark/spark-0.8.0-incubating/core/src/main/java/org/apache/spark/network/netty/FileClient.java > (at line 23) > [error] import io.netty.channel.ChannelFutureListener; > [error] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > [error] The import io.netty.channel.ChannelFutureListener is never used > [error] ---------- > [error] ---------- > [error] 3. WARNING in > /home/spark/spark-0.8.0-incubating/core/src/main/java/org/apache/spark/network/netty/FileServer.java > (at line 23) > [error] import io.netty.channel.Channel; > [error] ^^^^^^^^^^^^^^^^^^^^^^^^ > [error] The import io.netty.channel.Channel is never used > [error] ---------- > [error] ---------- > [error] 4. WARNING in > /home/spark/spark-0.8.0-incubating/core/src/main/scala/org/apache/spark/api/java/JavaSparkContextVarargsWorkaround.java > (at line 20) > [error] import java.util.Arrays; > [error] ^^^^^^^^^^^^^^^^ > [error] The import java.util.Arrays is never used > [error] ---------- > [error] ---------- > [error] 5. WARNING in > /home/spark/spark-0.8.0-incubating/core/src/main/scala/org/apache/spark/api/java/function/DoubleFlatMapFunction.java > (at line 30) > [error] public abstract class DoubleFlatMapFunction<T> extends > AbstractFunction1<T, Iterable<Double>> > [error] ^^^^^^^^^^^^^^^^^^^^^ > [error] The serializable class DoubleFlatMapFunction does not declare a > static final serialVersionUID field of type long > [error] ---------- > [error] 6. ERROR in > /home/spark/spark-0.8.0-incubating/core/src/main/scala/org/apache/spark/api/java/function/DoubleFlatMapFunction.java > (at line 36) > [error] public final Iterable<Double> apply(T t) { return call(t); } > [error] ^^^^^^^^^^ > [error] The method apply(T) of type DoubleFlatMapFunction<T> must override a > superclass method > [error] ---------- > [error] 6 problems (1 error, 5 warnings) > [error] (core/compile:compile) javac returned nonzero exit code > [error] Total time: 133 s, completed Nov 21, 2013 2:12:11 PM > [root@kserv06 spark-0.8.0-incubating]#
