Thomas Tauber-Marshall created IMPALA-9716:
----------------------------------------------
Summary: Add jitter to the exponential backoff in status reporting
Key: IMPALA-9716
URL: https://issues.apache.org/jira/browse/IMPALA-9716
Project: IMPALA
Issue Type: Improvement
Components: Distributed Exec
Affects Versions: Impala 4.0
Reporter: Thomas Tauber-Marshall
Assignee: Thomas Tauber-Marshall
When status reports fail, we use exponential backoff when retrying sending
them. However, currently the backoff is deterministic, leading to a thundering
herd problem where all of the backends for a particular query may try to report
at the same time, the coordinator is overwhelmed and rejects some of the rpcs,
then the backends all backoff by the same amount and retry sending at the same
time, leading the coordinator to be overwhelmed again.
We can help solve this by adding some random jitter to the exponential backoff
time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
