> Can the 0.8.1.1 client still talk to 0.8.0 versions of Kafka Yes it can.
"0.8.1 is fully compatible with 0.8." It is buried on this page: http://kafka.apache.org/documentation.html In addition to the pom version bump SPARK-2492 would bring the kafka streaming receiver (which was originally based on kafka 0.7) in line with kafka 0.8: https://issues.apache.org/jira/browse/SPARK-2492 https://github.com/apache/spark/pull/1420 I will soon test that PR on a spark+yarn cluster Cheers, Sean On Nov 10, 2014, at 11:58 AM, Matei Zaharia <matei.zaha...@gmail.com<mailto:matei.zaha...@gmail.com>> wrote: Just curious, what are the pros and cons of this? Can the 0.8.1.1 client still talk to 0.8.0 versions of Kafka, or do you need it to match your Kafka version exactly? Matei On Nov 10, 2014, at 9:48 AM, Bhaskar Dutta <bhas...@gmail.com<mailto:bhas...@gmail.com>> wrote: Hi, Is there any plan to bump the Kafka version dependency in Spark 1.2 from 0.8.0 to 0.8.1.1? Current dependency is still on Kafka 0.8.0 https://github.com/apache/spark/blob/branch-1.2/external/kafka/pom.xml Thanks Bhaskie
