Did this on a fresh install - still same error.
On Wed, Jan 22, 2014 at 10:46 AM, Matei Zaharia <[email protected]>wrote: > Try doing a sbt clean before rebuilding. > > Matei > > On Jan 22, 2014, at 10:22 AM, Manoj Samel <[email protected]> > wrote: > > See thread below. Reposted as compilation error thread > > ---------- Forwarded message ---------- > From: Manoj Samel <[email protected]> > Date: Wed, Jan 22, 2014 at 10:20 AM > Subject: Re: make-distribution.sh error > org.apache.hadoop#hadoop-client;2.0.0: not found > To: [email protected] > > > I think I found the way to go beyond error "unresolved dependency: > org.apache.hadoop#hadoop-client;2.0.0: not found" > > On my machine, the command "hadoop version" gives "2.0.0-cdh4.5.0" , not > "2.0.0". > > After I changed the SPARK_HADOOP_VERSION to 2.0.0-cdh4.5.0, it is > proceeding beyond that point. > > However, now it fails with multiple compilation errors in > /.../streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala > > [error] > /data/spark/spark-0.8.1-incubating/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:605: > type mismatch; > [error] found : org.apache.spark.streaming.DStream[(K, (V, > com.google.common.base.Optional[W]))] > [error] required: > org.apache.spark.streaming.api.java.JavaPairDStream[K,(V, > com.google.common.base.Optional[W])] > [error] Note: implicit method fromPairDStream is not applicable here > because it comes after the application point and it lacks an explicit > result type > [error] joinResult.mapValues{case (v, w) => (v, > JavaUtils.optionToOptional(w))} > [error] ^ > [error] > /data/spark/spark-0.8.1-incubating/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:617: > type mismatch; > [error] found : org.apache.spark.streaming.DStream[(K, > (com.google.common.base.Optional[V], W))] > [error] required: > org.apache.spark.streaming.api.java.JavaPairDStream[K,(com.google.common.base.Optional[V], > W)] > [error] Note: implicit method fromPairDStream is not applicable here > because it comes after the application point and it lacks an explicit > result type > [error] joinResult.mapValues{case (v, w) => > (JavaUtils.optionToOptional(v), w)} > [error] ^ > [error] > /data/spark/spark-0.8.1-incubating/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:632: > type mismatch; > [error] found : org.apache.spark.streaming.DStream[(K, > (com.google.common.base.Optional[V], W))] > [error] required: > org.apache.spark.streaming.api.java.JavaPairDStream[K,(com.google.common.base.Optional[V], > W)] > [error] Note: implicit method fromPairDStream is not applicable here > because it comes after the application point and it lacks an explicit > result type > [error] joinResult.mapValues{case (v, w) => > (JavaUtils.optionToOptional(v), w)} > [error] ^ > [error] > /data/spark/spark-0.8.1-incubating/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:647: > type mismatch; > [error] found : org.apache.spark.streaming.DStream[(K, > (com.google.common.base.Optional[V], W))] > [error] required: > org.apache.spark.streaming.api.java.JavaPairDStream[K,(com.google.common.base.Optional[V], > W)] > [error] Note: implicit method fromPairDStream is not applicable here > because it comes after the application point and it lacks an explicit > result type > [error] joinResult.mapValues{case (v, w) => > (JavaUtils.optionToOptional(v), w)} > [error] ^ > [error] 43 errors found > [error] (streaming/compile:compile) Compilation failed > > > > > On Wed, Jan 22, 2014 at 10:09 AM, Manoj Samel <[email protected]>wrote: > >> Hi, >> >> On my cluster I have CDH 4.5 (Hadoop 2.0.0) installed. I installed spark >> 0.8.1 and tried to run make-distribution.sh after >> setting SPARK_HADOOP_VERSION=2.0.0 >> >> I get >> >> [warn] >> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client/2.0.0/hadoop-client-2.0.0.pom >> [info] Resolving org.apache.derby#derby;10.4.2.0 ... >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >> [warn] :: UNRESOLVED DEPENDENCIES :: >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >> [warn] :: org.apache.hadoop#hadoop-client;2.0.0: not found >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >> sbt.ResolveException: unresolved dependency: >> org.apache.hadoop#hadoop-client;2.0.0: not found >> at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:214) >> at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:122) >> at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:121) >> at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:117) >> at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:117) >> >> Any thoughts? >> > > > >
