[
https://issues.apache.org/jira/browse/HDFS-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15020866#comment-15020866
]
Vinayakumar B commented on HDFS-9038:
-------------------------------------
bq. The calculation in FsVolumeImpl#getNonDfsUsed looks wrong. Won't it always
be zero?
Do you mean, nonDfsUsed will be zero? from the logs and tests, nonDFSUsage, is
giving not zero.
>From the earlier table, Brahma posted, Jenkins machine had 2887306715136 bytes
>of non-dfs usage out of 23318276997120 capacity, which seems reasonable.
bq. The correct calculation should replace getAvailable() with actual volume
free space. Something like
Files.getFileStore(currentDir.toPath()).getUnallocatedSpace() perhaps.
IMO, This is also good point. Though earlier calculation was not entirely
wrong, which basically calculates the nonDfsUsed by subtracting available space
and dfsUsed from capacity. Available space is based on
{{File.getUsableSpace()}} api, which according to javadoc, usable by this JVM
not of entire partition.
Even though, I dont have any objection to use,
{{Files.getFileStore(currentDir.toPath()).getUnallocatedSpace()}}, how about
using {{currentDir.getFreeSpace()}}? Do you find any difference between them?
Updated calculation might look like this
{code}public long getNonDfsUsed() throws IOException {
long totalFreeSpace = currentDir.getFreeSpace();
return getCapacity() - getDfsUsed() - totalFreeSpace;
}{code}
Is this looks okay [~arpitagarwal] ?
> Reserved space is erroneously counted towards non-DFS used.
> -----------------------------------------------------------
>
> Key: HDFS-9038
> URL: https://issues.apache.org/jira/browse/HDFS-9038
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.7.1
> Reporter: Chris Nauroth
> Assignee: Brahma Reddy Battula
> Attachments: HDFS-9038-002.patch, HDFS-9038-003.patch,
> HDFS-9038-004.patch, HDFS-9038.patch
>
>
> HDFS-5215 changed the DataNode volume available space calculation to consider
> the reserved space held by the {{dfs.datanode.du.reserved}} configuration
> property. As a side effect, reserved space is now counted towards non-DFS
> used. I don't believe it was intentional to change the definition of non-DFS
> used. This issue proposes restoring the prior behavior: do not count
> reserved space towards non-DFS used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
