Re: [EXTERNAL] [DISCUSS] KIP-310: Add a Kafka Source Connector to Kafka Connect

2018-09-27 Thread Konstantine Karantasis
I agree with you Rhys that Kafka Connect is an integral part of Apache Kafka and it perfectly makes sense, in many cases, not to overload the core or the clients with responsibilities that are related to data export to and data import from specific systems. That can be true even when Kafka itself

Re: [EXTERNAL] [DISCUSS] KIP-310: Add a Kafka Source Connector to Kafka Connect

2018-09-26 Thread McCaig, Rhys
Hi Konstantine, Thank you for your thoughtful comments! > However, I don't think the apache/kafka repository is the right place to > host such a Connector. > I find this approach very appealing. AK focuses on providing the core > infrastructure for Connect, that is required in every Kafka

Re: [EXTERNAL] [DISCUSS] KIP-310: Add a Kafka Source Connector to Kafka Connect

2018-09-26 Thread Konstantine Karantasis
Hi Rhys, thanks for the proposal and apologies for the late feedback. Utilizing Connect to mirror Kafka topics is definitely a plausible proposal for a very useful use case. However, I don't think the apache/kafka repository is the right place to host such a Connector. Currently, no

Re: [EXTERNAL] [DISCUSS] KIP-310: Add a Kafka Source Connector to Kafka Connect

2018-08-04 Thread McCaig, Rhys
Hi All, If there are no further comments on this KIP I’ll start a vote early this week. Rhys On Aug 1, 2018, at 12:32 AM, McCaig, Rhys mailto:rhys_mcc...@cable.comcast.com>> wrote: Hi All, I’ve updated the proposal to include the improvements suggested by Stephane. I have also submitted a

Re: [EXTERNAL] [DISCUSS] KIP-310: Add a Kafka Source Connector to Kafka Connect

2018-08-01 Thread McCaig, Rhys
Hi All, I’ve updated the proposal to include the improvements suggested by Stephane. I have also submitted a PR to implement this functionality into Kafka. https://github.com/apache/kafka/pull/5438 I don’t have a benchmark against MirrorMaker yet, as I only currently have a local docker stack

Re: [EXTERNAL] [DISCUSS] KIP-310: Add a Kafka Source Connector to Kafka Connect

2018-06-18 Thread McCaig, Rhys
Hi Stephane, Thanks for your feedback and apologies for the delay in my response. > Are there any performance benchmarks against Mirror Maker available? I'm > interested to know if this is more performant / scalable. > Regarding the implementation, here's some feedback: Currently I don’t have