[
https://issues.apache.org/jira/browse/HDFS-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037733#comment-15037733
]
Brahma Reddy Battula commented on HDFS-9038:
--------------------------------------------
bq. I don't think switching to File#getFreeSpace is necessarily wrong, just
different from the pre-HDFS-5215 calculation, which might cause some surprises.
Yes, its surprising.
Difference in HDFS user about that {{~5%}} space,
before patch --> It was taken as already used by someone (non-dfs) because its
not usable by HDFS.
after patch --> It will display actual usage of someone (non-dfs) even though
its not usable.
Only matter is, value of metric. Which doesnt affect HDFS functionality anyway.
But assumption in calculattion in tests {{capacityUsed + capacityRemaining +
capacityUsedNonDFS == capacityTotal}} might need to change, as
{{capacityRemaining}} doesnt include, that {{~5%}}.
>From the above Test Failure of {{TestNameNodeMetrics}}, below values have got
|CapacityTotal|23318276997120|
|CapacityUsed |147456|
|CapacityRemaining |20273827930112|
|CapacityUsedNonDFS |1859806781440|
|| Difference || ~5% ||
*Now we have two options to go ahead with the patch.*
* Update the tests with assumption of extra {{~5-6%}} space for privileged users
* fallback to {{File.getUsablespace()}}
Which one do you prefer [~cnauroth],[~arpitagarwal]..?
> Reserved space is erroneously counted towards non-DFS used.
> -----------------------------------------------------------
>
> Key: HDFS-9038
> URL: https://issues.apache.org/jira/browse/HDFS-9038
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.7.1
> Reporter: Chris Nauroth
> Assignee: Brahma Reddy Battula
> Attachments: HDFS-9038-002.patch, HDFS-9038-003.patch,
> HDFS-9038-004.patch, HDFS-9038-005.patch, HDFS-9038.patch
>
>
> HDFS-5215 changed the DataNode volume available space calculation to consider
> the reserved space held by the {{dfs.datanode.du.reserved}} configuration
> property. As a side effect, reserved space is now counted towards non-DFS
> used. I don't believe it was intentional to change the definition of non-DFS
> used. This issue proposes restoring the prior behavior: do not count
> reserved space towards non-DFS used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
