> On Sept. 6, 2012, 6:14 a.m., Mike Percy wrote: > > Awesome... we definitely need exponential backoff on the load balancing > > sink processor. > > > > Only thing is, I wonder if we shouldn't just add it to the two existing > > selectors instead of implement new ones that essentially do the same thing. > > How about modifying the constructors to accept a boolean (backoffOnFailure) > > instead? > > > > Other than that it looks good, I saw only a couple minor edge case issues > > noted below.
Note: if we parameterize the constructors then of course we would add an additional config option to enable backoff on failure. - Mike ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6939/#review11083 ----------------------------------------------------------- On Sept. 6, 2012, 4:15 a.m., Juhani Connolly wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/6939/ > ----------------------------------------------------------- > > (Updated Sept. 6, 2012, 4:15 a.m.) > > > Review request for Flume. > > > Description > ------- > > Added a callback to the SinkSelector interface to inform selectors of sink > failures, adding a noop stub to the abstract source so existing selectors > don't need changes. > > The new selector maintains counts of sequential failures as well as times for > intended recovery and last failure time, which it uses to decide whether or > not a sink should be added to the returned iterator. The iterator is > generated in such a way that the round robin remains balanced. > > > This addresses bug FLUME-1541. > https://issues.apache.org/jira/browse/FLUME-1541 > > > Diffs > ----- > > flume-ng-core/src/main/java/org/apache/flume/sink/AbstractSinkSelector.java > 63397a5 > > flume-ng-core/src/main/java/org/apache/flume/sink/LoadBalancingSinkProcessor.java > 18d4509 > > flume-ng-core/src/test/java/org/apache/flume/sink/TestLoadBalancingSinkProcessor.java > 1e9c94e > > Diff: https://reviews.apache.org/r/6939/diff/ > > > Testing > ------- > > Added 3 new tests to verify that dynamic rebalancing is done, that the sink > becomes available again after timeout, and that the timeout increases with > sequential failures > > All tests pass > > > Thanks, > > Juhani Connolly > >
