[
https://issues.apache.org/jira/browse/HDFS-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996340#comment-15996340
]
maobaolong edited comment on HDFS-9038 at 5/4/17 8:29 AM:
----------------------------------------------------------
{code}
public long getNonDfsUsed() throws IOException {
long actualNonDfsUsed = getActualNonDfsUsed();
if (actualNonDfsUsed < reserved) {
return 0L;
}
return actualNonDfsUsed - reserved;
}
{code}
The code block above is the function to caculate nonDfsUsed, but in fact it
will let the result to be 0L out of expect. Such as this following situation:
du.reserved = 50G
Disk Capacity = 2048G
Disk Available = 2000G
Dfs used = 30G
usage.getUsed() = dirFile.getTotalSpace() - dirFile.getFreeSpace()
= 2048G - 2000G
= 48G
getActualNonDfsUsed = usage.getUsed() - getDfsUsed()
= 48G - 30G
= 18G
18G < 50G, so the function `getNonDfsUsed` actualNonDfsUsed < reserved, and the
NonDfsUsed will return 0, is that logic make sense?
was (Author: maobaolong):
```java
public long getNonDfsUsed() throws IOException {
long actualNonDfsUsed = getActualNonDfsUsed();
if (actualNonDfsUsed < reserved) {
return 0L;
}
return actualNonDfsUsed - reserved;
}
```
The code block above is the function to caculate nonDfsUsed, but in fact it
will let the result to be 0L out of expect. Such as this following situation:
du.reserved = 50G
Disk Capacity = 2048G
Disk Available = 2000G
Dfs used = 30G
usage.getUsed() = dirFile.getTotalSpace() - dirFile.getFreeSpace()
= 2048G - 2000G
= 48G
getActualNonDfsUsed = usage.getUsed() - getDfsUsed()
= 48G - 30G
= 18G
18G < 50G, so the function `getNonDfsUsed` actualNonDfsUsed < reserved, and the
NonDfsUsed will return 0, is that logic make sense?
> DFS reserved space is erroneously counted towards non-DFS used.
> ---------------------------------------------------------------
>
> Key: HDFS-9038
> URL: https://issues.apache.org/jira/browse/HDFS-9038
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.7.1
> Reporter: Chris Nauroth
> Assignee: Brahma Reddy Battula
> Fix For: 2.8.0, 2.7.4, 3.0.0-alpha2
>
> Attachments: GetFree.java, HDFS-9038-002.patch, HDFS-9038-003.patch,
> HDFS-9038-004.patch, HDFS-9038-005.patch, HDFS-9038-006.patch,
> HDFS-9038-007.patch, HDFS-9038-008.patch, HDFS-9038-009.patch,
> HDFS-9038-010.patch, HDFS-9038-011.patch, HDFS-9038-branch-2.7.patch,
> HDFS-9038.patch
>
>
> HDFS-5215 changed the DataNode volume available space calculation to consider
> the reserved space held by the {{dfs.datanode.du.reserved}} configuration
> property. As a side effect, reserved space is now counted towards non-DFS
> used. I don't believe it was intentional to change the definition of non-DFS
> used. This issue proposes restoring the prior behavior: do not count
> reserved space towards non-DFS used.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
