Hi, XiaoChuan, Just confirmed with Jiangjie Qin (our in-house Kafka PMC), Kafka client 0.10.1.1 does not work with broker 0.10.0 due to new protocol version introduced in 0.10.1.1. Hence, if you try to run Samza 0.13.1 against Kafka broker 0.10.0 version, it won't work. The best option for that seems to be swapping out the runtime Kafka client 0.10.1.1 version in Samza 0.13.1 to Kafka client 0.10.0.1 (which was used in Samza 0.12). Since there is no incompatible API calls to Kafka client lib between Samza 0.12.0 and Samza 0.13.1, that should work and is worth trying out.
Let us know if we can be further assistance here. Thanks! -Yi On Thu, Aug 31, 2017 at 4:14 PM, Yi Pan <nickpa...@gmail.com> wrote: > Hi, Xiaochuan, > > I am relaying your question to our in-house Kafka expert to confirm. But > as far as I can tell from the description, the main breaking changes are in > the KStreams API. Samza only uses the standard Kafka consumer/producer APIs > now and should not be affected. > > I would recommend to give it a try in your question 1. > > As for your question 2, are you referring to a) binding a Kafka 0.10.0.0 > client library with Samza 0.13.1? Or b) running Samza 0.13.1 against Kafka > broker version 0.10.0.0? > > If you are referring to a), from the commit history, there is no source > code level changes when we upgrade the dependency from Kafka 0.10.0.1 to > 0.10.1.1 in Samza 0.13.1. If you want to make sure that works, you can try > to downgrade the Kafka version in your build environment to 0.10.0.0 and > build your app to see whether it breaks the build. If the build passes, the > client should work w/ Samza 0.13.1, unless Kafka client has an API that > changes runtime behavior between 0.10.0.1 and 0.10.1.1. > > If you are referring to b), it is not guaranteed that the higher version > of Kafka client library will work with an older version of Kafka broker. > However, Samza has been very conservative in the usage of new features in > Kafka client library. Hence, you would have a higher chance of success to > run Samza 0.13.1 against Kafka 0.10.0 brokers. > > Unfortunately, we have not test all the above combinations internally. I > would wait for the confirmation from our Kafka expert before I can give you > a definite answer. > > Thanks! > > -Yi >
