[
https://issues.apache.org/jira/browse/SAMZA-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897327#comment-13897327
]
Jakob Homan commented on SAMZA-30:
----------------------------------
My suggestion was more to encapsulate all of backoff logic in a separate class
with just a single method, sleep, that would do the everything that's currently
done in your patch in the brokerproxy itself.
{noformat}
class ExpoBackOff() {
// all the vals
var sleepBackOffMs = /* What I was referring to as sleep history */
def sleep() = { // all the logic from getNextDeliveryDelay and the actual
sleeping }
}
{noformat}
That way the only change to brokerproxy would be to create the ExpoBackOff
instance and call {{ebo.sleep}}. This separates out the useful backoff
function into something we can re-use elsewhere and minimizes extra,
non-broker-proxying logic in BrokerProxy itself.
> Back off in BrokerProxy when fetch fails
> ----------------------------------------
>
> Key: SAMZA-30
> URL: https://issues.apache.org/jira/browse/SAMZA-30
> Project: Samza
> Issue Type: Bug
> Components: kafka
> Affects Versions: 0.6.0
> Reporter: Chris Riccomini
> Assignee: Steve Yates
> Labels: newbie
> Attachments: SAMZA-30.diff
>
>
> The BrokerProxy immediately recreates its simple consumer, and re-fetches
> when an exception is thrown in its fetch loop (inside its thread). We should
> add a backoff here. Rather than doing a fixed time (say sleep 10s), let's do
> an exponential backoff with a cap (0s, 100s, 200s, 400s, 800s, 1.6s, .. up to
> cap), so we don't introduce lag in the case where there's a one-off failure
> and an immediate reconnect would work.
> I suggest adding a config to allow the backoff time and max cap and to be
> defined, but default it to 100ms, and 10s, respectively.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
