[
https://issues.apache.org/jira/browse/HADOOP-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas updated HADOOP-4430:
------------------------------------
Release Note:
Incompatible changes:
1) Config parameter dfs.datanode.du.pct is no longer used and is removed from
the hadoop-default.xml.
2) Namenoe Web UI has the following changes:
The following parameters are removed:
* Total Capacity
The following parameters are added to both Cluster Summary and Datanode
information:
* Configured Capacity - This is total diskspace of all the data directories
minus the resereved capacity defined by
* Non DFS Used - This indicates the disk space taken by non DFS file
* DFS remaining % - This is remaining % of Configured Capacity available for
DFS use
The following parameters are modified:
* DFS Used % - This is changed from % of Total Capacity to % of Configured
Capacity
Hadoop Flags: [Incompatible change]
Status: Patch Available (was: Open)
> Namenode Web UI capacity report is inconsistent with Balancer
> -------------------------------------------------------------
>
> Key: HADOOP-4430
> URL: https://issues.apache.org/jira/browse/HADOOP-4430
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4430.patch
>
>
> Solution to 2816 changed
> - Total Capacity definition from (the disk space of all data directories) to
> (the disk space of all the data directories - the reserved space)
> - We added a new element Present Capacity to the report. It is set to (Used
> Capacity + Remaining Capacity)
> - We changed the Used Percentage reported from (Used Capacity)/(Total
> Capacity) to (Used Capacity)/(Present Capacity)
> - All these changes are displayed on Namenode Web UI.
> Balancer functionality
> Balancer script is started with a threshold parameter. It tries to move the
> blocks from the nodes that have Used % that is more than (Cluster average +
> threshold) to the nodes that have less than (Cluster average - threshold).
> Essentially balancer gets all the datanodes used % to with in (the Cluster
> average +/- threshold).
> Inconsistencies due to the change in 2816
> When MapReduce jobs are run, temporary files are generated. This eats away a
> lot of space from Present Capacity. The difference between the Total Capacity
> and the Present Capacity can be huge. Currently balancer computes Used
> Percentage based (Used Capacity)/(Total Capacity). The Used % the balancer
> uses could be significantly different from Used % displayed on the Namenode
> Web UI. When balancer is done balancing, the Namenode Used % might still
> appear unbalanced.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
