Yida Wu has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/21896 )
Change subject: IMPALA-12146: Fix incorrect host memory reserved when the executor quits abnormally ...................................................................... IMPALA-12146: Fix incorrect host memory reserved when the executor quits abnormally Currently there is an issue where if an executor quits abnormally while running a query, its reserved memory may still remain in the coordinator's host stats. The problem happens when the statestore sends a topic update to udpate the host stats. While the coordinator removes the remote stats for the terminated executor during the update, it does not remove the corresponding stats from the host stats, which store aggregated information for all hosts. This can lead to the stale stats remaining. To fix this, added a logic to ensure that the stats are properly removed from the host stats when a delete topic is detected and the host no longer exists in any remote pool stats. Tests: Passed core tests. Added testcase AdmissionControllerTest::EraseHostStats. Manually verified that the coordinator web ui correctly showed the reserved memory after the crashed executor recovered and rejoined. Change-Id: Ic6f6edd28c55904d63d0c494230ee2bf7a0f6cce --- M be/src/scheduling/admission-controller-test.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h 3 files changed, 119 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/21896/2 -- To view, visit http://gerrit.cloudera.org:8080/21896 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic6f6edd28c55904d63d0c494230ee2bf7a0f6cce Gerrit-Change-Number: 21896 Gerrit-PatchSet: 2 Gerrit-Owner: Yida Wu <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
