[
https://issues.apache.org/jira/browse/HDFS-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15035067#comment-15035067
]
Chris Nauroth commented on HDFS-9038:
-------------------------------------
bq. Brahma changed this at my suggestion. Exposing reserved bytes through
nonDfsUsed was inadvertent side effect of HDFS-6898.
Sorry, I missed that earlier comment. I understand the rationale now, so I'll
withdraw my comment about changing it.
Regarding {{File#getUsableSpace}} vs. {{File#getFreeSpace}}, I've been
approaching this as a regression introduced by HDFS-5215, with the goal of
restoring the pre-HDFS-5215 behavior as closely as possible. Before HDFS-5215,
non-DFS used was based on {{File#getUsableSpace}}, indirectly because of use of
{{getAvailable}} in the calculation. This meant that "free but unusable" space
would have counted towards non-DFS used.
However, that would then lead me to expect seeing a non-zero non-DFS used for
the typical 5% reserved for privileged users. In practice, I haven't seen that
happen though. I'm not sure why not.
I tried digging through revision history for past decisions. The use of
{{File#getUsableSpace}} traces back to HADOOP-5958, when the code switched from
forking to {{df}} to using then-new JDK 1.6 APIs. There is a question posted
about which method to use, but no specific discussion about why
{{File#getUsableSpace}} was chosen.
I don't think switching to {{File#getFreeSpace}} is necessarily wrong, just
different from the pre-HDFS-5215 calculation, which might cause some surprises.
> Reserved space is erroneously counted towards non-DFS used.
> -----------------------------------------------------------
>
> Key: HDFS-9038
> URL: https://issues.apache.org/jira/browse/HDFS-9038
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.7.1
> Reporter: Chris Nauroth
> Assignee: Brahma Reddy Battula
> Attachments: HDFS-9038-002.patch, HDFS-9038-003.patch,
> HDFS-9038-004.patch, HDFS-9038-005.patch, HDFS-9038.patch
>
>
> HDFS-5215 changed the DataNode volume available space calculation to consider
> the reserved space held by the {{dfs.datanode.du.reserved}} configuration
> property. As a side effect, reserved space is now counted towards non-DFS
> used. I don't believe it was intentional to change the definition of non-DFS
> used. This issue proposes restoring the prior behavior: do not count
> reserved space towards non-DFS used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
