Yida Wu has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/21896 )
Change subject: IMPALA-12146: Fix incorrect host memory reserved when the executor quits abnormally ...................................................................... IMPALA-12146: Fix incorrect host memory reserved when the executor quits abnormally Currently there is an issue where if an executor quits abnormally while running a query, its reserved memory may still remain in the coordinator's host stats. The problem happens when the statestore sends a topic update to update the host stats. Although the coordinator removes the remote stats for the terminated executor during the update, it doesn't reset the aggregated memory reserved in the host stats if all remote stats for that executor have been removed. This can lead to stale stats remaining. To fix this, added a logic to ensure that the stats of memory reserved of that host are reset in the aggregated host stats when a delete topic is detected and the host no longer exists in any remote pool stats. Tests: Passed exhaustive tests. Added testcase AdmissionControllerTest::EraseHostStats. Manually verified that the coordinator web ui correctly showed the reserved memory after the crashed executor recovered and rejoined. Change-Id: Ic6f6edd28c55904d63d0c494230ee2bf7a0f6cce --- M be/src/scheduling/admission-controller-test.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h 3 files changed, 116 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/21896/3 -- To view, visit http://gerrit.cloudera.org:8080/21896 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic6f6edd28c55904d63d0c494230ee2bf7a0f6cce Gerrit-Change-Number: 21896 Gerrit-PatchSet: 3 Gerrit-Owner: Yida Wu <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Andrew Sherman <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
