That PR hasn't been updated since the new kafka streaming stuff (including KafkaCluster) got merged to master, it will require more changes than what's in there currently.
On Tue, Feb 10, 2015 at 9:25 AM, Sean Owen <so...@cloudera.com> wrote: > Yes, did you see the PR for SPARK-2808? > https://github.com/apache/spark/pull/3631/files > > It requires more than just changing the version. > > On Tue, Feb 10, 2015 at 3:11 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > Compiling Spark master branch against Kafka 0.8.2, I got: > > > > [WARNING] > > > /home/hbase/spark/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala:62: > > no valid targets for annotation on value ssc_ - it is discarded unused. > > You may specify targets with meta-annotations, e.g. @(transient @param) > > [WARNING] @transient ssc_ : StreamingContext, > > [WARNING] ^ > > [ERROR] > > > /home/hbase/spark/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaCluster.scala:273: > > type mismatch; > > found : > > Map[kafka.common.TopicAndPartition,kafka.common.OffsetMetadataAndError] > > required: > > > scala.collection.immutable.Map[kafka.common.TopicAndPartition,kafka.common.OffsetAndMetadata] > > [ERROR] val req = OffsetCommitRequest(groupId, metadata) > > [ERROR] ^ > > [ERROR] > > > /home/hbase/spark/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaCluster.scala:278: > > value requestInfo is not a member of kafka.api. > > OffsetCommitResponse > > [ERROR] val respMap = resp.requestInfo > > [ERROR] ^ > > > > FYI > > > > On Tue, Feb 10, 2015 at 3:23 AM, Sean Owen <so...@cloudera.com> wrote: > >> > >> I believe Kafka 0.8.2 is imminent for master / 1.4.0 -- see SPARK-2808. > >> Anecdotally I have used Spark 1.2 with Kafka 0.8.2 without problem > already. > >> > >> On Feb 10, 2015 10:53 AM, "critikaled" <isasmani....@gmail.com> wrote: > >>> > >>> When can we expect the latest kafka and scala 2.11 support in spark > >>> streaming? > >>> > >>> > >>> > >>> -- > >>> View this message in context: > >>> > http://apache-spark-user-list.1001560.n3.nabble.com/Kafka-Version-Update-0-8-2-status-tp21573.html > >>> Sent from the Apache Spark User List mailing list archive at > Nabble.com. > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > >>> For additional commands, e-mail: user-h...@spark.apache.org > >>> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
