Re: [DISCUSS] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-14 Thread Matthias J. Sax
Thanks for the feedback Bruno. I updated the KIP (I thought the info was there, but it seems it was too implicit). 1) Each time a client TimeoutException happens, we would log a WARN message. If `task.timeout.ms` expires, we would rethrow the last client `TimeoutException` to stop processing.

Re: [DISCUSS] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-14 Thread John Roesler
Thanks for the update, Matthias! Other than Bruno’s good points, this proposal looks good to me. Thanks, John On Thu, May 14, 2020, at 07:17, Bruno Cadonna wrote: > Hi Matthias, > > Thank you for the KIP. I like your KIP. > > Here my feedback: > > 1. The KIP is not clear about what should

Re: [DISCUSS] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-14 Thread Bruno Cadonna
Hi Matthias, Thank you for the KIP. I like your KIP. Here my feedback: 1. The KIP is not clear about what should happen when task.timeout.ms expires. To facilitate the mapping from the error users might encounter due to timeouts to this KIP, it would be good to state the error that will be

Re: [DISCUSS] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-12 Thread Matthias J. Sax
John, Guozhang, thanks a lot for your feedback. I updated the KIP on a slightly different angle: instead of using retries, we should switch to a timeout based approach. I also extended the KIP to deprecate producer/admin `retries` config. I like the proposal to skip a task if a client

Re: [DISCUSS] KIP-572: Improve timeouts and retires in Kafka Streams

2020-02-27 Thread Guozhang Wang
Hello John, I'll make note that you owe me a beer now :) I think I'm leaning towards your approach as well based on my observations on previously reported timeout exceptions in the past. I once left some thoughts on Matthias' PR here

Re: [DISCUSS] KIP-572: Improve timeouts and retires in Kafka Streams

2020-02-27 Thread John Roesler
Hi Matthias, Thanks for the proposal! I think this will be a wonderful improvement to Streams. In particular, thanks for the motivation. It would indeed be nice not to have to set long timeout configs and block individual client requests in order to cope with transient slow responses. I'm very

[DISCUSS] KIP-572: Improve timeouts and retires in Kafka Streams

2020-02-21 Thread Matthias J. Sax
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi, I would like to propose KIP-572 to make Kafka Streams more robust with regard to timeout exception handling. https://cwiki.apache.org/confluence/display/KAFKA/KIP-572%3A+Improve+tim eouts+and+retries+in+Kafka+Streams Note, that there is a