Thanks for the confirmation, Ismael. I will write something up for further discussion.
-Dana On Jul 5, 2016 4:43 PM, "Ismael Juma" <ism...@juma.me.uk> wrote: > Hi Dana, > > Thanks for the PR. Technically, a (simple) KIP is required when proposing > new configs. > > Ismael > > On Sun, Jun 19, 2016 at 7:42 PM, Dana Powers <dana.pow...@gmail.com> > wrote: > > > I took a stab at implementing a simplified exponential + randomized > > backoff policy here: https://github.com/apache/kafka/pull/1523 > > > > Rather than changing public interfaces / using plugins, it just adds a > > new client configuration "reconnect.backoff.max" that can be used to > > enable increasing backoff when node failures repeat. If this > > configuration is not set higher than reconnect.backoff.ms then the > > current constant backoff policy is retained. The default is to > > continue w/ current 50ms constant backoff. > > > > Thoughts? Would a change like this require a KIP? > > > > -Dana > > > > > > On Mon, May 2, 2016 at 10:29 AM, Guozhang Wang <wangg...@gmail.com> > wrote: > > > For the specific problem of connection storm, randomized with normal > > > distribution at specified mean as "reconnect.backoff.ms" has been > proved > > > pretty well. The most recent usage of it in my mind is RAFT, and it > turns > > > out pretty effective in eliminating leader-election storms. > > > > > > > > > Guozhang > > > > > > On Fri, Apr 29, 2016 at 8:57 PM, Ewen Cheslack-Postava < > > e...@confluent.io> > > > wrote: > > > > > >> I'll agree w/ Jay and point out that the implementations of > > >> ReconnectionPolicy provided built-in with that driver are Constant, > > >> Exponential, and Counting. Constant and exponential can be combined > with > > >> the right set of policy config parameters. I'm curious if there's a > real > > >> need for something else, or if you're just looking for something > > >> exponential instead of non-constant? I think a fixed exponential > backoff > > >> policy that defaults parameters to the current constant backoff policy > > >> would probably satisfy our needs. > > >> > > >> On Mon, Apr 4, 2016 at 1:25 PM, Jay Kreps <j...@confluent.io> wrote: > > >> > > >> > If I understand the problem we are fixing is a connection storm > where > > >> when > > >> > a new broker comes online it is overwhelmed with connections. > > >> > > > >> > In general we try hard to avoid plugins where possible. Maybe > instead > > of > > >> > adding another plugin interface we could just directly solve this > > problem > > >> > by doing some randomization in the backoff to space out the > > >> reconnections? > > >> > This seems like it would be good for anyone with a large client > > >> > environment? > > >> > > > >> > -Jay > > >> > > > >> > On Mon, Apr 4, 2016 at 12:54 PM, Florian Hussonnois < > > >> fhussonn...@gmail.com > > >> > > > > >> > wrote: > > >> > > > >> > > Hi Kafka Team, > > >> > > > > >> > > I have made a new Kafka Improvement Proposal. > > >> > > > > >> > > > > >> > > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-53+-+Add+custom+policies+for+reconnect+attempts+to+NetworkdClient > > >> > > > > >> > > This is my first proposal so I don't know if I have given enough > > >> > > information. > > >> > > Also I have already proposed an implementation : > > >> > > https://github.com/apache/kafka/pull/1179 > > >> > > > > >> > > Thanks > > >> > > > > >> > > -- > > >> > > Florian HUSSONNOIS > > >> > > > > >> > > > >> > > >> > > >> > > >> -- > > >> Thanks, > > >> Ewen > > >> > > > > > > > > > > > > -- > > > -- Guozhang > > >
