[
https://issues.apache.org/jira/browse/HDFS-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15057367#comment-15057367
]
Vinayakumar B commented on HDFS-9038:
-------------------------------------
[~arpitagarwal], thanks for jumping in for calculation.
I think considering We use {{File#getUsableSpace}} instead of
{{File#getFreeSpace}}, current code after HDFS-5215 also comes to same equation
as mentioned by [~arpitagarwal], brahma's patch doesn't change anything in
this, except subtracting 'reservedForReplicas' also.
But, I think what [~cnauroth] expects is,
*Since {{reserved}} is hidden from the HDFS in {{getCapacity()}} itself, we can
think that's already used for {{nonDfs}} and subtract it from actual ondisk
{{nonDfsUsage}}, showing only HDFS visible {{nonDfsUsage}}*. {{nonDfsUsage}}
metric will be positive, only if the ondisk nonDfsUsage crosses beyond
{{reserved}}, and it shows only excess usage beyond {{reserved}}.
i.e. nonDfsUsage will be {{2GB}} instead of {{3GB}} in my earlier example,
where {{1GB}} was reserved.
So final equation would be {code}nonDfsUsage=File#getTotalSpace - dfsUsed -
File#getUsableSpace - reserved{code}
Code would look like this.
{code}
public long getNonDfsUsed() throws IOException {
long nonDfsUsed = getCapacity() - getDfsUsed() - getAvailable() - reserved
- getReservedForReplicas();
return (nonDfsUsed > 0) ? nonDfsUsed : 0;
}
{code}
Am I right [~cnauroth]?
> DFS reserved space is erroneously counted towards non-DFS used.
> ---------------------------------------------------------------
>
> Key: HDFS-9038
> URL: https://issues.apache.org/jira/browse/HDFS-9038
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.7.1
> Reporter: Chris Nauroth
> Assignee: Brahma Reddy Battula
> Attachments: HDFS-9038-002.patch, HDFS-9038-003.patch,
> HDFS-9038-004.patch, HDFS-9038-005.patch, HDFS-9038-006.patch,
> HDFS-9038-007.patch, HDFS-9038.patch
>
>
> HDFS-5215 changed the DataNode volume available space calculation to consider
> the reserved space held by the {{dfs.datanode.du.reserved}} configuration
> property. As a side effect, reserved space is now counted towards non-DFS
> used. I don't believe it was intentional to change the definition of non-DFS
> used. This issue proposes restoring the prior behavior: do not count
> reserved space towards non-DFS used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
