Hi Vino,
please don't interpret my opinion as some official project decision.
For discussions like this I would definitely prefer to hear the opinions of
others in the project.
Perhaps having a new client API and having compatibility layers inside the
connector would be another option.
So per default the compatibility level of the Kafka client lib would be used
but a developer could explicitly choose
older compatibility levels, where we have taken care of the work to decide what
works and what doesn't.
Chris
Am 12.03.18, 13:07 schrieb "vino yang" <yanghua1...@gmail.com>:
Hi Chris,
In some ways, I argee with you. Though kafka API has the compatibility. But
- old API + higher server version : this mode would miss some key new
feature.
- new API + older server version : this mode, users are in a puzzle
about which feature they could use and which could not. Also, new API
will
do more logic judgement and something else (which cause performance cost)
for backward compatibility.
I think it's the main reason that other framework split different kafka
connector with versions.
Anyway, I will respect your decision. Can I claim this task about upgrading
the kafka client's version to 1.x?
2018-03-12 16:30 GMT+08:00 Christofer Dutz <christofer.d...@c-ware.de>:
> Hi Vino,
>
> I would rather go a different path. I talked to some Kafka pros and they
> sort of confirmed my gut-feeling.
> The greatest changes to Kafka have been in the layers behind the API
> itself. The API seems to have been designed with backward compatibility in
> mind.
> That means you can generally use a newer API with an older broker as well
> as use a new broker with an older API (This is probably even the safer way
> around). As soon as you try to do something with the API which your broker
> doesn't support, you get error messages.
>
> https://cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix
>
> I would rather update the existing connector to a newer Kafka version ...
> 0.8.2.2 is quite old and we should update to a version of at least 0.10.0
> (I would prefer a 1.x) and stick with that. I doubt many will be using an
> ancient 0.8.2 version (09.09.2015). And everything starting with 0.10.x
> should be interchangeable.
>
> I wouldn't like to have yet another project maintaining a Zoo of adapters
> for Kafka.
>
> Eventually a Kafka-Streams client would make sense though ... to sort of
> extend the Edgent streams from the edge to the Kafka cluster.
>
> Chris
>
>
>
> Am 12.03.18, 03:41 schrieb "vino yang" <yanghua1...@gmail.com>:
>
> Hi guys,
>
> How about this idea, I think we should support kafka's new client API.
>
> 2018-03-04 15:10 GMT+08:00 vino yang <yanghua1...@gmail.com>:
>
> > The reason is that Kafka 0.9+ provided a new consumer API which has
> more
> > features and better performance.
> >
> > Just like Flink's implementation : https://github.com/apache/
> > flink/tree/master/flink-connectors.
> >
> > vinoyang
> > Thanks.
> >
> >
>
>
>
