[
https://issues.apache.org/jira/browse/HDFS-17206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ke Han updated HDFS-17206:
--------------------------
Description:
When running hdfs 3.3.6 cluster, I noticed that the quota doesn't work properly.
I have a file whose size is 32KB, and I set the quota to 80 KB. First, I upload
it to the hdfs, there should still be 48 KB of space left. But when I try to
append the same file to the previous file, it failed with exception
*appendToFile: Quota by storage type : DISK on path : /GvvSzRSH is exceeded.
quota = 80 KB but space consumed = 255.94 MB*
This is the first file I uploaded, so there should only be 32KB be used. But it
prompts {*}255.94MB{*}.
h1. Reproduce
Start up a 4 node hdfs-3.3.6 cluster (latest release version), execute the
following commands
{code:java}
bin/hdfs dfs -mkdir /GvvSzRSH
bin/hdfs dfsadmin -setSpaceQuota 81920 -storageType DISK /GvvSzRSH/
bin/hdfs dfsadmin -setSpaceQuota 81920 -storageType PROVIDED /GvvSzRSH/
bin/hdfs dfs -put -p /tmp/upfuzz/hdfs/dBJShMmu/G /GvvSzRSH/
bin/hdfs dfs -appendToFile /tmp/dBJShMmu/G /GvvSzRSH/G
# appendToFile: Quota by storage type : DISK on path : /GvvSzRSH is exceeded.
quota = 80 KB but space consumed = 255.94 MB
bin/hdfs dfs -count -q -h -v -x -t DISK /GvvSzRSH/
# q or u option is given, the -x option is ignored.
# DISK_QUOTA REM_DISK_QUOTA PATHNAME
# 80 K 80 K /GvvSzRSH{code}
/tmp/dBJShMmu/G is a file with size 32B. I have attached the dir to this
failure report.
This issue also happen at 3.2.4 hdfs cluster.
was:
When running hdfs 3.3.6 cluster, I noticed that the quota doesn't work properly.
I have a file whose size is 32KB, and I set the quota to 80 KB. First, I upload
it to the hdfs, there should still be 48 KB of space left. But when I try to
append the same file to the previous file, it failed with exception
*appendToFile: Quota by storage type : DISK on path : /GvvSzRSH is exceeded.
quota = 80 KB but space consumed = 255.94 MB*
This is the first file I uploaded, so there should only be 32KB be used. But it
prompts {*}255.94MB{*}.
h1. Reproduce
Start up a 4 node hdfs-3.3.6 cluster (latest release version), execute the
following commands
{code:java}
bin/hdfs dfs -mkdir /GvvSzRSH
bin/hdfs dfsadmin -setSpaceQuota 81920 -storageType DISK /GvvSzRSH/
bin/hdfs dfsadmin -setSpaceQuota 81920 -storageType PROVIDED /GvvSzRSH/
bin/hdfs dfs -put -p /tmp/upfuzz/hdfs/dBJShMmu/G /GvvSzRSH/
bin/hdfs dfs -appendToFile /tmp/dBJShMmu/G /GvvSzRSH/G
# appendToFile: Quota by storage type : DISK on path : /GvvSzRSH is exceeded.
quota = 80 KB but space consumed = 255.94 MB
bin/hdfs dfs -count -q -h -v -x -t DISK /GvvSzRSH/
# q or u option is given, the -x option is ignored.
# DISK_QUOTA REM_DISK_QUOTA PATHNAME
# 80 K 80 K /GvvSzRSH{code}
/tmp/dBJShMmu/G is a file with size 32B. I have attached the dir to this
failure report.
> Quota not working when append a file for HDFS-3.3.6 cluster
> -----------------------------------------------------------
>
> Key: HDFS-17206
> URL: https://issues.apache.org/jira/browse/HDFS-17206
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 3.2.4, 3.3.6
> Reporter: Ke Han
> Priority: Major
> Attachments: dBJShMmu.tar.gz
>
>
> When running hdfs 3.3.6 cluster, I noticed that the quota doesn't work
> properly.
> I have a file whose size is 32KB, and I set the quota to 80 KB. First, I
> upload it to the hdfs, there should still be 48 KB of space left. But when I
> try to append the same file to the previous file, it failed with exception
> *appendToFile: Quota by storage type : DISK on path : /GvvSzRSH is exceeded.
> quota = 80 KB but space consumed = 255.94 MB*
> This is the first file I uploaded, so there should only be 32KB be used. But
> it prompts {*}255.94MB{*}.
> h1. Reproduce
> Start up a 4 node hdfs-3.3.6 cluster (latest release version), execute the
> following commands
> {code:java}
> bin/hdfs dfs -mkdir /GvvSzRSH
> bin/hdfs dfsadmin -setSpaceQuota 81920 -storageType DISK /GvvSzRSH/
> bin/hdfs dfsadmin -setSpaceQuota 81920 -storageType PROVIDED /GvvSzRSH/
> bin/hdfs dfs -put -p /tmp/upfuzz/hdfs/dBJShMmu/G /GvvSzRSH/
> bin/hdfs dfs -appendToFile /tmp/dBJShMmu/G /GvvSzRSH/G
> # appendToFile: Quota by storage type : DISK on path : /GvvSzRSH is exceeded.
> quota = 80 KB but space consumed = 255.94 MB
> bin/hdfs dfs -count -q -h -v -x -t DISK /GvvSzRSH/
> # q or u option is given, the -x option is ignored.
> # DISK_QUOTA REM_DISK_QUOTA PATHNAME
> # 80 K 80 K /GvvSzRSH{code}
> /tmp/dBJShMmu/G is a file with size 32B. I have attached the dir to this
> failure report.
>
> This issue also happen at 3.2.4 hdfs cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
