[
https://issues.apache.org/jira/browse/HDFS-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053971#comment-15053971
]
Vinayakumar B commented on HDFS-9038:
-------------------------------------
bq. dfs.datanode.du.reserved means that the administrator intentionally wants
to hold back a portion of the volume, essentially making it invisible to HDFS.
This is the same reason that dfs.datanode.du.reserved gets subtracted from the
capacity calculation. This is why I think 3 GB is incorrect in the example
above. Non-DFS usage is really "unexpected non-DFS usage". By setting
dfs.datanode.du.reserved to 1 GB, you have stated that up to 1 GB of usage by
something other than HDFS is expected, so it's incorrect to count it as part of
"unexpected non-DFS usage".
So you mean, when 1GB is configured as reserved, 'Non-DFS' usage metric should
show 2GB, even though actual usage by files other than HDFS is 3GB.?
I was thinking 'NonDfsUsage' should show actual non-dfs usage. Not only
unexpected part of it. So if actual non-dfs usage is within this reserved
limit, metric should show 0. right?
In that case, wouldn't it be better to rename the 'NonDfsUsage' metric to
'UnexpectedNonDfsUsage' to make it more clear? Or mention somewhere to clear
confusion in people like me.
> DFS reserved space is erroneously counted towards non-DFS used.
> ---------------------------------------------------------------
>
> Key: HDFS-9038
> URL: https://issues.apache.org/jira/browse/HDFS-9038
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.7.1
> Reporter: Chris Nauroth
> Assignee: Brahma Reddy Battula
> Attachments: HDFS-9038-002.patch, HDFS-9038-003.patch,
> HDFS-9038-004.patch, HDFS-9038-005.patch, HDFS-9038-006.patch,
> HDFS-9038-007.patch, HDFS-9038.patch
>
>
> HDFS-5215 changed the DataNode volume available space calculation to consider
> the reserved space held by the {{dfs.datanode.du.reserved}} configuration
> property. As a side effect, reserved space is now counted towards non-DFS
> used. I don't believe it was intentional to change the definition of non-DFS
> used. This issue proposes restoring the prior behavior: do not count
> reserved space towards non-DFS used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
