Balancer sometimes runs out of memory after days or weeks running
-----------------------------------------------------------------
Key: HADOOP-5145
URL: https://issues.apache.org/jira/browse/HADOOP-5145
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Reporter: Hairong Kuang
Assignee: Hairong Kuang
Fix For: 0.21.0
The culprit is a HashMap called MovedBlocks. By design this map does not get
cleaned up between iterations. This is because the deletion of source replicas
is done by NN. When next iteration starts, source replicas may not have been
deleted, Balancer does not want to schedule them to move again. To prevent
running out of memory, Balancer should expire/clean the movedBlocks from some
iterations back.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
