Re: BEAM-307(KafkaIO on Kafka 0.10)

checking the interface of 0.8.2, 0.9.0, 0.10.0. Eventually the type is unified as Collection<> from mixing vararg, List<>, Collection<>, On Wed, Feb 8, 2017 at 12:33 PM, Raghu Angadi wrote: > On Wed, Feb 8, 2017 at 12:19 PM, Jesse Anderson >

Re: BEAM-307(KafkaIO on Kafka 0.10)

On Wed, Feb 8, 2017 at 12:19 PM, Jesse Anderson wrote: > I'm not. There was a decent amount of time between the first 0.8 and 0.9 > release. > The ones that affect are minor changes between 0.9 and 0.10 (e.g. change vararg to Collection<>). May be both could have existed

Re: BEAM-307(KafkaIO on Kafka 0.10)

True. I was commenting on Kafka developers. I am surprised the api breakages didn't have any deprecation period at all. On Wed, Feb 8, 2017 at 12:02 PM, Xu Mingmin wrote: > i tend to have more versions supported, actually in our prod environment, > there're 0.8, 0.9 and

Re: BEAM-307(KafkaIO on Kafka 0.10)

I've a proposal design doc here https://docs.google.com/document/d/1YlCWws4SYCqUWAtVz9mrmVdFNM8b3E8DLCImYep-I6k/edit# which try to share the same code of KafkaIO. Due to the API change of Kafka Consumer.class, it's nearly impossible to have a single module for both kafka 0.9 and 0.10, unless

Re: BEAM-307(KafkaIO on Kafka 0.10)

This would be a great contribution, Mingmin! As a general rule, we'd like the connector to work with as many versions as possible, with as little code duplication as possible. Slightly orthogonal -- BigtableIO [1] is an example of a connector that separates API portions from the underlying

Re: BEAM-307(KafkaIO on Kafka 0.10)

I see. kafka-clients dependency could also be in 'provided' scope so that is simpler to use different versions at runtime. On Mon, Feb 6, 2017 at 12:05 PM, Xu Mingmin wrote: > The one I meet is external authentication added in 0.10, we use a > standalone token-based security

Re: BEAM-307(KafkaIO on Kafka 0.10)

Current KafkaIO works just fine with Kafka 0.10. I don't know of any incompatibilities or regressions. It does not take advantage of message timestamps, of course. It would be good to take handle tme in in a backward compatible way.. it might be required anyway if they are optional in 0.10. Not