[
https://issues.apache.org/jira/browse/HDFS-14764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xun REN updated HDFS-14764:
---------------------------
Attachment: hdfs_count_withsnapshot.txt
> HDFS count doesn't include snapshot files correctly
> ---------------------------------------------------
>
> Key: HDFS-14764
> URL: https://issues.apache.org/jira/browse/HDFS-14764
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Xun REN
> Priority: Major
> Attachments: hdfs_count_withsnapshot.txt
>
>
> Hi,
>
> When we set a quota on a path, and that path contains some snapshots, in this
> case, the status shown by the command "hdfs dfs -count -v -q /my_path"
> doesn't match the real quota usage.
> The -count here will only count the current path without counting the files
> in the snapshots which are already deleted in the current path.
> If there is a job continues to write files into that path, it will report an
> error like
> {code:java}
> The NameSpace quota (directories and files) of directory /my_path is
> exceeded{code}
> While the count command shows there is still space.
> Because, when we write files into a directory, it will also check the
> snapshot files. But the count command will not check.
>
> The idea here is to modify the report of "hdfs dfs -count" to include also
> the files in snapshots. Ideally, we could add an additional column to show
> the total number of files of the current directory + files deleted from the
> current directory but referenced in the snapshots.
>
> You could find in the attached text file the steps to reproduce the issue.
>
> Thanks.
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
