Compiling Spark master branch against Kafka 0.8.2, I got: [WARNING] /home/hbase/spark/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala:62: no valid targets for annotation on value ssc_ - it is discarded unused. You may specify targets with meta-annotations, e.g. @(transient @param) [WARNING] @transient ssc_ : StreamingContext, [WARNING] ^ [ERROR] /home/hbase/spark/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaCluster.scala:273: type mismatch; found : Map[kafka.common.TopicAndPartition,kafka.common.OffsetMetadataAndError] required: scala.collection.immutable.Map[kafka.common.TopicAndPartition,kafka.common.OffsetAndMetadata] [ERROR] val req = OffsetCommitRequest(groupId, metadata) [ERROR] ^ [ERROR] /home/hbase/spark/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaCluster.scala:278: value requestInfo is not a member of kafka.api. OffsetCommitResponse [ERROR] val respMap = resp.requestInfo [ERROR] ^
FYI On Tue, Feb 10, 2015 at 3:23 AM, Sean Owen <[email protected]> wrote: > I believe Kafka 0.8.2 is imminent for master / 1.4.0 -- see SPARK-2808. > Anecdotally I have used Spark 1.2 with Kafka 0.8.2 without problem already. > On Feb 10, 2015 10:53 AM, "critikaled" <[email protected]> wrote: > >> When can we expect the latest kafka and scala 2.11 support in spark >> streaming? >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Kafka-Version-Update-0-8-2-status-tp21573.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >>
