Great.. that makes sense. Forward compatibility by brokers is likely
hard, tho it would be nice if clients were backward compatible. I
guess, tho, implementing that requires KIP-35.
Thanks for the 0.9.0.0 rolling update pointer.
Richard
On 10/01/2015 10:48 AM, Grant Henke wrote:
Hi Richard,
You are correct that version will now be 0.9.0 and anything referencing
0.8.3 is being changed. You are also correct in the there have been wire
protocol changes that break compatibility. However, backwards compatibility
exists and you should always upgrade your brokers before upgrading your
clients in order to avoid issues (In the future KIP-35
<http://KIP-35+-+Retrieving+protocol+version> may change that).
It's also worth noting that if you are performing a rolling upgrade of your
brokers, you need to be sure brokers running the new protocol know to
communicate with the old version to remain compatible during the bounce.
This is done using the inter.broker.protocol.version property. More on that
topic can be read here:
https://kafka.apache.org/083/documentation.html#upgrade
Hopefully that helps clear things up.
Thank you,
Grant
On Thu, Oct 1, 2015 at 12:21 PM, Richard Lee <[email protected]> wrote:
Note the 0.8.3-SNAPSHOT has recently been renamed 0.9.0.0-SNAPSHOT.
In any event, the major version number change could indicate that there
has, in fact, been some sort of incompatible change. Using 0.9.0.0, I'm
also unable to use the kafka-console-consumer.sh to read from a 0.8.2.1
broker, but it works fine with a 0.9.0.0 broker.
Some validation from a kafka expert that broker forward compatibility (or
client backward compatibility) is not supported would be appreciated, and
that this isn't just a case of some sort of local, fixable misconfiguration.
Thanks!
Richard
On 09/30/2015 11:17 AM, Doug Tomm wrote:
hello,
i've got a set of broker nodes running 0.8.2.1. on my laptop i'm also
running 0.8.2.1, and i have a single broker node and mirrormaker there.
i'm also using kafka-console-consumer.sh on the mac to display messages on
a favorite topic being published from the broker nodes. there are no
messages on the topic, but everything is well-behaved. i can inject
messages with kafkacat and everything is fine.
but then!
on the laptop i switched everything to 0.8.3 but left the broker nodes
alone. now when i run mirrormaker i see this:
[2015-09-30 10:44:55,090] WARN
[ConsumerFetcherThread-tivo_kafka_110339-mbpr.local-1443635093396-c55cbafb-0-5],
Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@61cb11c5.
Possible cause: java.nio.BufferUnderflowException
(kafka.consumer.ConsumerFetcherThread)
[2015-09-30 10:44:55,624] WARN
[ConsumerFetcherThread-tivo_kafka_110339-mbpr.local-1443635093396-c55cbafb-0-5],
Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@3c7bb986.
Possible cause: java.nio.BufferUnderflowException
(kafka.consumer.ConsumerFetcherThread)
[2015-09-30 10:44:56,181] WARN
[ConsumerFetcherThread-tivo_kafka_110339-mbpr.local-1443635093396-c55cbafb-0-5],
Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@1d4fbd2c.
Possible cause: java.nio.BufferUnderflowException
(kafka.consumer.ConsumerFetcherThread)
[2015-09-30 10:44:56,726] WARN
[ConsumerFetcherThread-tivo_kafka_110339-mbpr.local-1443635093396-c55cbafb-0-5],
Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@59e67b2f.
Possible cause: java.nio.BufferUnderflowException
(kafka.consumer.ConsumerFetcherThread)
if i use kafkacat to generate a message on the topic i see
IllegalArgumentExceptions instead.
this suggests that the two versions of kafka aren't compatible. is this
the case? does the whole ecosystem need to be on the same version?
thank you,
doug
