Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15860 )
Change subject: IMPALA-9716: Add jitter to the exponential backoff in status reporting ...................................................................... Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/15860/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/15860/1//COMMIT_MSG@20 PS1, Line 20: Testing: > I've been thinking about how to test more directly for the "thundering herd Yeah, I think we can verify the formula by inspection and at least the code path is covered. I can't see a way to deterministically test that it solves the problem with 3 executors. http://gerrit.cloudera.org:8080/#/c/15860/1/be/src/runtime/query-state.cc File be/src/runtime/query-state.cc: http://gerrit.cloudera.org:8080/#/c/15860/1/be/src/runtime/query-state.cc@564 PS1, Line 564: return report_interval * (num_failed_reports_ + 1 + jitter); What do you think about spreading out the reports even more? E.g. something like report_interval * (1 + num_failed_reports_ * jitter) -- To view, visit http://gerrit.cloudera.org:8080/15860 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id05c224517aa606057117328f480dfa98676b923 Gerrit-Change-Number: 15860 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall <[email protected]> Gerrit-Reviewer: David Rorke <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Mon, 04 May 2020 22:50:30 +0000 Gerrit-HasComments: Yes
