Hello David Rorke, Tim Armstrong, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15860
to look at the new patch set (#2).
Change subject: IMPALA-9716: Add jitter to the exponential backoff in status
reporting
......................................................................
IMPALA-9716: Add jitter to the exponential backoff in status reporting
When status reports fail, we use exponential backoff when retrying
sending them. However, currently the backoff is deterministic, leading
to a thundering herd problem where all of the backends for a
particular query may try to report at the same time, the coordinator
is overwhelmed and rejects some of the rpcs, then the backends all
backoff by the same amount and retry sending at the same time, leading
the coordinator to be overwhelmed again.
This patch alleviates this problem by adding some random jitter to the
exponential backoff used when a status report fails.
Testing:
- Passed a full run of existing tests.
- Code path is covered by test_reportexecstatus_retries
Change-Id: Id05c224517aa606057117328f480dfa98676b923
---
M be/src/runtime/query-state.cc
1 file changed, 9 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/15860/2
--
To view, visit http://gerrit.cloudera.org:8080/15860
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id05c224517aa606057117328f480dfa98676b923
Gerrit-Change-Number: 15860
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall <[email protected]>
Gerrit-Reviewer: David Rorke <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
