eliaslevy commented on issue #6577: [FLINK-7964] Add Apache Kafka 1.0/1.1 connectors URL: https://github.com/apache/flink/pull/6577#issuecomment-417387441 @aljoscha * [KIP-35 - Retrieving protocol version](https://cwiki-test.apache.org/confluence/display/KAFKA/KIP-35+-+Retrieving+protocol+version) * [KIP-97: Improved Kafka Client RPC Compatibility Policy](https://cwiki-test.apache.org/confluence/display/KAFKA/KIP-97%3A+Improved+Kafka+Client+RPC+Compatibility+Policy) > This KIP suggests revising the Kafka Java client compatibility policy so that it is two-way, rather than one-way. Not only should older brokers support newer clients, but newer clients should support older brokers. > In some cases, older brokers will be unable to perform a feature supported by newer clients. For example, retrieving message offsets from a message timestamp (KIP-79) is not supported in versions before 0.10.1. In these cases, the new client will throw an ObsoleteBrokerVersion exception when attempting to perform the operation. This will let clients know that the reason why the operation cannot be performed is because the broker version in use does not support it. At the same time, clients which do not attempt to use the new operations will proceed as before. * [Compatibility Matrix](https://cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix) Basically, any KIP-35 client can talk to a Kafka broker >= 0.10.0, which for the Java client [means](https://cwiki.apache.org/confluence/display/KAFKA/KIP-35+-+Retrieving+protocol+version) versions >= 0.10.2. @pnowojski I am suggesting that, except possibly for 0.8 and 0.9 clients, there is only a need for a single Kafka connector based on the latest Kafka client (2.0.0 as of now). There is no need for separate 2.0.0, 1.x.y, 0.11.0.x, and 0.10.x.y connectors. A connector based on 2.0.0 can talk to all those brokers.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
