Re: Seeking Feedback on Kafka Connect Issues

2018-04-02 Thread Matt Farmer
Howdy, I'll keep an eye on that ticket. KIP-275 came out of some work we've done internally on our private forks of Kafka and the Confluent Cloud Storage connector. Essentially, with that extra API we've tweaked the S3 connector to check the value of isClosing in preCommit and immediately attempt

Re: Seeking Feedback on Kafka Connect Issues

2018-04-02 Thread Randall Hauch
Yes, Confluent would be interested in improvements to the S3 connector. Feel free to create an issue/PR in https://github.com/confluentinc/kafka-connect-storage-cloud/. I just created https://issues.apache.org/jira/browse/KAFKA-6738 to deal with the bad data handling issue, and we can use that to

Re: Seeking Feedback on Kafka Connect Issues

2018-03-20 Thread Matt Farmer
Hi Ewen, Thanks for the thoughtful response. I’m happy to take some time to write up a KIP and do some implementation work here. I did KIP-210 previously, so I’ve been through the process before. We also have some organizational interest for improving Kafka Connect. Our concern internally is

Re: Seeking Feedback on Kafka Connect Issues

2018-03-19 Thread Ewen Cheslack-Postava
Responses inline. On Mon, Mar 19, 2018 at 3:02 PM, Matt Farmer wrote: > Hi everyone, > > We’ve been experimenting recently with some limited use of Kafka Connect > and are hoping to expand to wider use cases soon. However, we had some > internal issues that gave us a well-timed

Seeking Feedback on Kafka Connect Issues

2018-03-19 Thread Matt Farmer
Hi everyone, We’ve been experimenting recently with some limited use of Kafka Connect and are hoping to expand to wider use cases soon. However, we had some internal issues that gave us a well-timed preview of error handling behavior in Kafka Connect. I think the fixes for this will require at