Hi Vino,

please don't interpret my opinion as some official project decision. 
For discussions like this I would definitely prefer to hear the opinions of 
others in the project. 
Perhaps having a new client API and having compatibility layers inside the 
connector would be another option.
So per default the compatibility level of the Kafka client lib would be used 
but a developer could explicitly choose 
older compatibility levels, where we have taken care of the work to decide what 
works and what doesn't. 

Chris 



´╗┐Am 12.03.18, 13:07 schrieb "vino yang" <yanghua1...@gmail.com>:

    Hi Chris,
    
    In some ways, I argee with you. Though kafka API has the compatibility. But
    
    
       - old API + higher server version : this mode would miss some key new
       feature.
       - new API + older server version : this mode, users are in a puzzle
       about which feature they could use and which could not. Also, new API 
will
       do more logic judgement and something else (which cause performance cost)
       for backward compatibility.
    
    I think it's the main reason that other framework split different kafka
    connector with versions.
    
    Anyway, I will respect your decision. Can I claim this task about upgrading
    the kafka client's version to 1.x?
    
    
    2018-03-12 16:30 GMT+08:00 Christofer Dutz <christofer.d...@c-ware.de>:
    
    > Hi Vino,
    >
    > I would rather go a different path. I talked to some Kafka pros and they
    > sort of confirmed my gut-feeling.
    > The greatest changes to Kafka have been in the layers behind the API
    > itself. The API seems to have been designed with backward compatibility in
    > mind.
    > That means you can generally use a newer API with an older broker as well
    > as use a new broker with an older API (This is probably even the safer way
    > around). As soon as you try to do something with the API which your broker
    > doesn't support, you get error messages.
    >
    > https://cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix
    >
    > I would rather update the existing connector to a newer Kafka version ...
    > 0.8.2.2 is quite old and we should update to a version of at least 0.10.0
    > (I would prefer a 1.x) and stick with that. I doubt many will be using an
    > ancient 0.8.2 version (09.09.2015). And everything starting with 0.10.x
    > should be interchangeable.
    >
    > I wouldn't like to have yet another project maintaining a Zoo of adapters
    > for Kafka.
    >
    > Eventually a Kafka-Streams client would make sense though ... to sort of
    > extend the Edgent streams from the edge to the Kafka cluster.
    >
    > Chris
    >
    >
    >
    > Am 12.03.18, 03:41 schrieb "vino yang" <yanghua1...@gmail.com>:
    >
    >     Hi guys,
    >
    >     How about this idea, I think we should support kafka's new client API.
    >
    >     2018-03-04 15:10 GMT+08:00 vino yang <yanghua1...@gmail.com>:
    >
    >     > The reason is that Kafka 0.9+ provided a new consumer API which has
    > more
    >     > features and better performance.
    >     >
    >     > Just like Flink's implementation : https://github.com/apache/
    >     > flink/tree/master/flink-connectors.
    >     >
    >     > vinoyang
    >     > Thanks.
    >     >
    >     >
    >
    >
    >
    

Reply via email to