I know this does not address the larger problem, but in this specific case, would the 0.4.1 Kafka NAR still work in 0.5.x?
If the NAR doesn't depend on any other NARs I would think it would still work, and could be a work around for those that need to stay on Kafka 0.8.2. On Sunday, February 21, 2016, Oleg Zhurakousky <[email protected]> wrote: > The unfortunate part is that between 0.8 and 0.9 there are also breaking > API changes on the Kafka side that would affect our code, so I say we need > to probably start thinking more about versioning. And in fact we are in the > concept of extension registry, but what I am now suggesting is that > versioning must come in isolation from registry. > > > As far as Zookeeper, i simply pointed out as one of the changes that were > made., so I am glad it’s not affecting it. I guess to leaves the protocol > incompatibilities. > > Cheers > Oleg > > On Feb 21, 2016, at 5:23 PM, Joe Witt <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > > Yeah the intent is to support 0.8 and 0.9. Will figure something out. > > Thanks > Joe > On Feb 21, 2016 4:47 PM, "West, Joshua" <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > >> Hi Oleg, >> >> Hmm -- from what I can tell, this isn't a Zookeeper communication issue. >> Nifi is able to connect into the Kafka brokers' Zookeeper cluster and >> retrieve the list of the kafka brokers to connect to. Seems, from the >> logs, to be a problem when attempting to consume from Kafka itself. >> >> I'm guessing that the Kafka 0.9.0 client libraries just are not >> compatible with Kafka 0.8.2.1 so in order to use Nifi 0.5.0 with Kafka, the >> Kafka version must be >= 0.9.0. >> >> Any change Nifi could add backwards compatible support for Kafka 0.8.2.1 >> too? Let you choose which client library version, when setting up the >> GetKafka processor? >> >> -- >> Josh West <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> >> Bose Corporation >> >> >> >> On Sun, 2016-02-21 at 15:02 +0000, Oleg Zhurakousky wrote: >> >> Josh >> >> Also, keep in mind that there are incompatible property names in Kafka >> between the 0.7 and 0.8 releases. One of the change that went it was >> replacing “zk.connectiontimeout.ms” with “zookeeper.connection.timeout.ms”. >> Not sure if it’s related though, but realizing that 0.4.1 was relying on >> this property it’s value was completely ignored with 0.8 client libraries >> (you could actually see the WARN message to that effect) and now it is not >> ignored, so take a look and see if tinkering with its value changes >> something. >> >> Cheers >> Oleg >> >> On Feb 20, 2016, at 6:47 PM, Oleg Zhurakousky < >> [email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> >> Josh >> >> The only change that ’s went and relevant to your issue is the fact that >> we’ve upgraded client libraries to Kafka 0.9 and between 0.8 and 0.9 Kafka >> introduced wire protocol changes that break compatibility. >> I am still digging so stay tuned. >> >> Oleg >> >> On Feb 20, 2016, at 4:10 PM, West, Joshua <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> >> Hi Oleg and Joe, >> >> Kafka 0.8.2.1 >> >> Attached is the app log with hostnames scrubbed. >> >> Thanks for your help. Much appreciated. >> >> -- >> Josh West <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> >> Bose Corporation >> >> >> >> On Sat, 2016-02-20 at 15:46 -0500, Joe Witt wrote: >> >> And also what version of Kafka are you using? >> On Feb 20, 2016 3:37 PM, "Oleg Zhurakousky" <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> >> Josh >> >> Any chance to attache the app-log or relevant stack trace? >> >> Thanks >> Oleg >> >> On Feb 20, 2016, at 3:30 PM, West, Joshua <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> >> Hi folks, >> >> I've upgraded from Nifi 0.4.1 to 0.5.0 and I am no longer able to use the >> GetKafka processor. I'm seeing errors like so: >> >> 2016-02-20 20:10:14,953 WARN >> [ConsumerFetcherThread-NiFi-sldjflkdsjflksjf_**SCRUBBED**-1455999008728-5b8c7108-0-0] >> kafka.consumer.ConsumerFetcherThread >> [ConsumerFetcherThread-NiFi-sldjflkdsjflksjf_**SCRUBBED**-1455999008728-5b8c7108-0-0], >> Error in fetchkafka.consumer.ConsumerFetcherThread$FetchRequest@7b49a642 >> <javascript:_e(%7B%7D,'cvml','kafka.consumer.ConsumerFetcherThread$FetchRequest@7b49a642');>. >> Possible cause: java.lang.IllegalArgumentException >> >> ^ Note the hostname of the server has been scrubbed. >> >> My configuration is pretty generic, except that with Zookeeper we use a >> different root path, so our Zookeeper connect string looks like so: >> >> zookeeper-node1:2181,zookeeper-node2:2181,zookeeper-node3:2181/kafka >> >> Is anybody else experiencing issues? >> >> Thanks. >> >> -- >> Josh West <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> >> >> Cloud Architect >> Bose Corporation >> >> >> >> >> >> <nifi-app.log.kafkaissues.bz2> >> >> >> >> > -- Sent from Gmail Mobile
