[
https://issues.apache.org/jira/browse/HDFS-6093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935758#comment-13935758
]
Arpit Agarwal edited comment on HDFS-6093 at 3/14/14 10:45 PM:
---------------------------------------------------------------
Hi Andrew,
I just tried this out your patch and I think there is some mismatch between the
output of {{dfsAdmin -report}} and {{cacheadmin -listPools}}.
This is with a single NN/single DN pseudocluster on Centos 6.5.
I ran the following commands:
- bin/hdfs cacheadmin -addPool pool1 -limit 1073741824
- bin/hdfs cacheadmin -addDirective -path /f1 -pool pool1
This says FILES_CACHED is zero.
{code}
$ bin/hdfs cacheadmin -listPools -stats
Found 1 result.
NAME OWNER GROUP MODE LIMIT MAXTTL BYTES_NEEDED
BYTES_CACHED BYTES_OVERLIMIT FILES_NEEDED FILES_CACHED
pool1 aagarwal aagarwal rwxr-xr-x 1073741824 never 1048576
0 0 1 0
{code}
However this says "cache used" is 1MB.
{code}
$ bin/hdfs dfsadmin -report
Configured Capacity: 49202208768 (45.82 GB)
Present Capacity: 39676268544 (36.95 GB)
DFS Remaining: 39675179008 (36.95 GB)
DFS Used: 1089536 (1.04 MB)
DFS Used%: 0.00%
Configured Cache Capacity: 268435456 (256 MB)
Present Cache Capacity: 268435456 (256 MB)
Cache Remaining: 267386880 (255 MB)
Cache Used: 1048576 (1 MB)
Cache Used%: 0.39%
{code}
I did not see any error messages related to caching in the DN/NN logs.
was (Author: arpitagarwal):
Hi Andrew,
I just tried this out your patch and I think there is some mismatch between the
output of {{dfsAdmin -report}} and {{cacheadmin -listPools}}.
This is with a single NN/single DN pseudocluster on Centos 6.5.
I ran the following commands:
- bin/hdfs cacheadmin -addPool pool1 -limit 1073741824
- bin/hdfs cacheadmin -addDirective -path /f1 -pool pool1
This says FILES_CACHED is zero.
{code}
$ bin/hdfs cacheadmin -listPools -stats
Found 1 result.
NAME OWNER GROUP MODE LIMIT MAXTTL BYTES_NEEDED
BYTES_CACHED BYTES_OVERLIMIT FILES_NEEDED FILES_CACHED
pool1 aagarwal aagarwal rwxr-xr-x 1073741824 never 1048576
0 0 1 0
{code}
However this says "cache used" is 1MB.
{code}
aagarwal@arrow ~/deploy2/hadoop-3.0.0-SNAPSHOT$ bin/hdfs dfsadmin -report
Configured Capacity: 49202208768 (45.82 GB)
Present Capacity: 39676268544 (36.95 GB)
DFS Remaining: 39675179008 (36.95 GB)
DFS Used: 1089536 (1.04 MB)
DFS Used%: 0.00%
Configured Cache Capacity: 268435456 (256 MB)
Present Cache Capacity: 268435456 (256 MB)
Cache Remaining: 267386880 (255 MB)
Cache Used: 1048576 (1 MB)
Cache Used%: 0.39%
{code}
I did not see any error messages related to caching in the DN/NN logs.
> Expose more caching information for debugging by users
> ------------------------------------------------------
>
> Key: HDFS-6093
> URL: https://issues.apache.org/jira/browse/HDFS-6093
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: caching
> Affects Versions: 2.4.0
> Reporter: Andrew Wang
> Assignee: Andrew Wang
> Attachments: hdfs-6093-1.patch
>
>
> When users submit a new cache directive, it's unclear if the NN has
> recognized it and is actively trying to cache it, or if it's hung for some
> other reason. It'd be nice to expose a "pending caching/uncaching" count the
> same way we expose pending replication work.
> It'd also be nice to display the aggregate cache capacity and usage in
> dfsadmin -report, since we already have have it as a metric and expose it
> per-DN in report output.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
