Try doing a sbt clean before rebuilding. Matei
On Jan 22, 2014, at 10:22 AM, Manoj Samel <[email protected]> wrote: > See thread below. Reposted as compilation error thread > > ---------- Forwarded message ---------- > From: Manoj Samel <[email protected]> > Date: Wed, Jan 22, 2014 at 10:20 AM > Subject: Re: make-distribution.sh error > org.apache.hadoop#hadoop-client;2.0.0: not found > To: [email protected] > > > I think I found the way to go beyond error "unresolved dependency: > org.apache.hadoop#hadoop-client;2.0.0: not found" > > On my machine, the command "hadoop version" gives "2.0.0-cdh4.5.0" , not > "2.0.0". > > After I changed the SPARK_HADOOP_VERSION to 2.0.0-cdh4.5.0, it is proceeding > beyond that point. > > However, now it fails with multiple compilation errors in > /.../streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala > > [error] > /data/spark/spark-0.8.1-incubating/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:605: > type mismatch; > [error] found : org.apache.spark.streaming.DStream[(K, (V, > com.google.common.base.Optional[W]))] > [error] required: org.apache.spark.streaming.api.java.JavaPairDStream[K,(V, > com.google.common.base.Optional[W])] > [error] Note: implicit method fromPairDStream is not applicable here because > it comes after the application point and it lacks an explicit result type > [error] joinResult.mapValues{case (v, w) => (v, > JavaUtils.optionToOptional(w))} > [error] ^ > [error] > /data/spark/spark-0.8.1-incubating/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:617: > type mismatch; > [error] found : org.apache.spark.streaming.DStream[(K, > (com.google.common.base.Optional[V], W))] > [error] required: > org.apache.spark.streaming.api.java.JavaPairDStream[K,(com.google.common.base.Optional[V], > W)] > [error] Note: implicit method fromPairDStream is not applicable here because > it comes after the application point and it lacks an explicit result type > [error] joinResult.mapValues{case (v, w) => > (JavaUtils.optionToOptional(v), w)} > [error] ^ > [error] > /data/spark/spark-0.8.1-incubating/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:632: > type mismatch; > [error] found : org.apache.spark.streaming.DStream[(K, > (com.google.common.base.Optional[V], W))] > [error] required: > org.apache.spark.streaming.api.java.JavaPairDStream[K,(com.google.common.base.Optional[V], > W)] > [error] Note: implicit method fromPairDStream is not applicable here because > it comes after the application point and it lacks an explicit result type > [error] joinResult.mapValues{case (v, w) => > (JavaUtils.optionToOptional(v), w)} > [error] ^ > [error] > /data/spark/spark-0.8.1-incubating/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:647: > type mismatch; > [error] found : org.apache.spark.streaming.DStream[(K, > (com.google.common.base.Optional[V], W))] > [error] required: > org.apache.spark.streaming.api.java.JavaPairDStream[K,(com.google.common.base.Optional[V], > W)] > [error] Note: implicit method fromPairDStream is not applicable here because > it comes after the application point and it lacks an explicit result type > [error] joinResult.mapValues{case (v, w) => > (JavaUtils.optionToOptional(v), w)} > [error] ^ > [error] 43 errors found > [error] (streaming/compile:compile) Compilation failed > > > > > On Wed, Jan 22, 2014 at 10:09 AM, Manoj Samel <[email protected]> > wrote: > Hi, > > On my cluster I have CDH 4.5 (Hadoop 2.0.0) installed. I installed spark > 0.8.1 and tried to run make-distribution.sh after setting > SPARK_HADOOP_VERSION=2.0.0 > > I get > > [warn] > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client/2.0.0/hadoop-client-2.0.0.pom > [info] Resolving org.apache.derby#derby;10.4.2.0 ... > [warn] :::::::::::::::::::::::::::::::::::::::::::::: > [warn] :: UNRESOLVED DEPENDENCIES :: > [warn] :::::::::::::::::::::::::::::::::::::::::::::: > [warn] :: org.apache.hadoop#hadoop-client;2.0.0: not found > [warn] :::::::::::::::::::::::::::::::::::::::::::::: > sbt.ResolveException: unresolved dependency: > org.apache.hadoop#hadoop-client;2.0.0: not found > at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:214) > at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:122) > at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:121) > at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:117) > at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:117) > > Any thoughts? > >
