Hi Dana, Thanks for the PR. Technically, a (simple) KIP is required when proposing new configs.
Ismael On Sun, Jun 19, 2016 at 7:42 PM, Dana Powers <dana.pow...@gmail.com> wrote: > I took a stab at implementing a simplified exponential + randomized > backoff policy here: https://github.com/apache/kafka/pull/1523 > > Rather than changing public interfaces / using plugins, it just adds a > new client configuration "reconnect.backoff.max" that can be used to > enable increasing backoff when node failures repeat. If this > configuration is not set higher than reconnect.backoff.ms then the > current constant backoff policy is retained. The default is to > continue w/ current 50ms constant backoff. > > Thoughts? Would a change like this require a KIP? > > -Dana > > > On Mon, May 2, 2016 at 10:29 AM, Guozhang Wang <wangg...@gmail.com> wrote: > > For the specific problem of connection storm, randomized with normal > > distribution at specified mean as "reconnect.backoff.ms" has been proved > > pretty well. The most recent usage of it in my mind is RAFT, and it turns > > out pretty effective in eliminating leader-election storms. > > > > > > Guozhang > > > > On Fri, Apr 29, 2016 at 8:57 PM, Ewen Cheslack-Postava < > e...@confluent.io> > > wrote: > > > >> I'll agree w/ Jay and point out that the implementations of > >> ReconnectionPolicy provided built-in with that driver are Constant, > >> Exponential, and Counting. Constant and exponential can be combined with > >> the right set of policy config parameters. I'm curious if there's a real > >> need for something else, or if you're just looking for something > >> exponential instead of non-constant? I think a fixed exponential backoff > >> policy that defaults parameters to the current constant backoff policy > >> would probably satisfy our needs. > >> > >> On Mon, Apr 4, 2016 at 1:25 PM, Jay Kreps <j...@confluent.io> wrote: > >> > >> > If I understand the problem we are fixing is a connection storm where > >> when > >> > a new broker comes online it is overwhelmed with connections. > >> > > >> > In general we try hard to avoid plugins where possible. Maybe instead > of > >> > adding another plugin interface we could just directly solve this > problem > >> > by doing some randomization in the backoff to space out the > >> reconnections? > >> > This seems like it would be good for anyone with a large client > >> > environment? > >> > > >> > -Jay > >> > > >> > On Mon, Apr 4, 2016 at 12:54 PM, Florian Hussonnois < > >> fhussonn...@gmail.com > >> > > > >> > wrote: > >> > > >> > > Hi Kafka Team, > >> > > > >> > > I have made a new Kafka Improvement Proposal. > >> > > > >> > > > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-53+-+Add+custom+policies+for+reconnect+attempts+to+NetworkdClient > >> > > > >> > > This is my first proposal so I don't know if I have given enough > >> > > information. > >> > > Also I have already proposed an implementation : > >> > > https://github.com/apache/kafka/pull/1179 > >> > > > >> > > Thanks > >> > > > >> > > -- > >> > > Florian HUSSONNOIS > >> > > > >> > > >> > >> > >> > >> -- > >> Thanks, > >> Ewen > >> > > > > > > > > -- > > -- Guozhang >
