Hello Jungtae, Thank you very much for your reply. Is it possible for us to used storm-kafka-client 1.1.0-SNAPSHOT in our fat jar, but deploy this fat jar with our existing Storm 1.0.1 cluster ; or do we need to also upgrade our cluster to 1.1.0-SNAPSHOT as well (I hope not) ?
Best regards, Alexandre 2017-01-12 9:48 GMT+01:00 Jungtaek Lim <[email protected]>: > Hi Alexandre, > > README of storm-kafka describes how to change Kafka version. > https://github.com/apache/storm/tree/master/external/ > storm-kafka#using-storm-kafka-with-different-versions-of-kafka > > As README stated, if you use Kafka 0.10.x, please try out new module > storm-kafka-client which is using new client API. > > storm-kafka-client 1.0.2 is compatible with Kafka 0.9, and > storm-kafka-client 1.1.0 SNAPSHOT is compatible with Kafka 0.10.x. > > I think we're close to start release process of Storm 1.1.0. Before that > you may need to build Storm 1.x branch via mvn install, and include > storm-kafka-client 1.1.0-SNAPSHOT as your fat jar. > > Thanks, > Jungtaek Lim > > 2017년 1월 12일 (목) 오후 12:30, Aaron Niskodé-Dossett <[email protected]>님이 작성: > > Hi Alexandre, > > My recollection from trying this when 0.10 was released is that it is > possible, but some small code changes are necessary. If you're comfortable > building Storm from source, updating the kafka version in the appropriate > poms and compiling would show you what needs to be updated. > > On Wed, Jan 11, 2017 at 2:20 PM Alexandre Vermeerbergen < > [email protected]> wrote: > > Hello, > > Any clue on whether or not it's possible to build a KafkaSpout / KafkaBolt > based on Apache Kafka 0.10.x ? > > Best regards, > Alexandre > > > 2017-01-07 18:03 GMT+01:00 Alexandre Vermeerbergen < > [email protected]>: > > Hello, > > We have been happyly running our real-time supervision homebrewed > application with Apache Storm 1.0.1 and its out-of-the-box StormKafka spout > & bold with our Kafka Broker 0.9.0.1 cluster for more than 6 months. > > We're been recently interested in upgrading our setup to Apache Storm > 1.0.2, mainly because we sometime hit https://issues.apache.org/ > jira/browse/STORM-1879 > > We also plan to soon upgrade our Kafka Broker cluster to 0.10.x > > However, we noticed that Storm 1.0.2's KafkaSpout has been downgraded to > Kafka 0.8.2.2 : STORM-1755: Revert the kafka client version upgrade in > storm-kafka module > > Kafka 0.10 brokers have a new message format which can be consumed and > produced to using older Kafka API, but at the price of performance > > Is it posible to build a KafkaStorm support based on Kafka 0.10.x API ? > > Best regards, > Alexandre > > >
