[
https://issues.apache.org/jira/browse/HDFS-13724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536726#comment-16536726
]
Hari Sekhon edited comment on HDFS-13724 at 7/9/18 9:36 AM:
------------------------------------------------------------
I would guess that something along the lines of this would be more intuitive to
users:
{code:java}
hdfs storagepolicies -listPaths{code}
as that would be more inline with the already existing command which lists just
the policies without paths:
{code:java}
hdfs storagepolicies -listPolicies{code}
Alternatively it could be a switch to -listPolicies such as (in a similar
fashion to what is done with hdfs fsck files blocks locations switches):
{code:java}
hdfs storagepolicies -listPolicies -paths{code}
[~brahmareddy]
{code:java}
grep -i storagepolicy fsimage.xml{code}
returns no hits. I actually had a colleague double check this for me last week
too, dumped all xml tags to sort uniq and there was no such tag or anything
that looked related, but there are definitely storage policies applied as my
hortonworks colleague who configured this one told me the path so I ran the
following command which showed the policy:
{code:java}
hdfs storagepolicies -getStoragePolicy -path /data/blah
The storage policy of /data/blah:
BlockStoragePolicy{COLD:2, storageTypes=[ARCHIVE], creationFallbacks=[],
replicationFallbacks=[]}
{code}
It was actually that discussion with him that made me realise that if he wasn't
around I wouldn't have been able to find the paths which had storage policies
applied to them, something that hadn't occurred to me the last time I was
configuring this myself.
was (Author: harisekhon):
I would guess that something along the lines of this would be more intuitive to
users:
{code:java}
hdfs storagepolicies -listPaths{code}
as that would be more inline with the already existing command which lists just
the policies without paths:
{code:java}
hdfs storagepolicies -listPolicies{code}
Alternatively it could be a switch to -listPolicies such as (in a similar
fashion to what is done with hdfs fsck files blocks locations switches):
{code:java}
hdfs storagepolicies -listPolicies -paths{code}
[~brahmareddy]
{code:java}
grep -i storagepolicy fsimage.xml{code}
returns no hits. I actually had a colleague double check this for me last week
too, dumped all xml tags to sort uniq and there was no such tag or anything
that looked related, but there are definitely storage policies applied as my
hortonworks colleague who configured this one told me the path and the
following command returns the policy:
{code:java}
hdfs storagepolicies -getStoragePolicy -path /data/blah
The storage policy of /data/blah:
BlockStoragePolicy{COLD:2, storageTypes=[ARCHIVE], creationFallbacks=[],
replicationFallbacks=[]}
{code}
> Storage Tiering Show Paths with Policies applied
> ------------------------------------------------
>
> Key: HDFS-13724
> URL: https://issues.apache.org/jira/browse/HDFS-13724
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Hari Sekhon
> Assignee: Yuanbo Liu
> Priority: Major
>
> Improvement Request to add an hdfs storagepolicies command to find paths for
> which storage policies have been applied.
> Right now you must explicitly query a single directory to get its policy to
> determine if one has been applied, but if another hadoop admin has configured
> policies on anything but trivially obvious paths such as /archive then there
> is no way to find which paths have policies applied to them other than by
> querying every single directory and subdirectory one by one which might
> potentially have a policy, eg:
> {code:java}
> hdfs storagepolicies -getStoragePolicy -path /dir3/subdir1
> hdfs storagepolicies -getStoragePolicy -path /dir2
> hdfs storagepolicies -getStoragePolicy -path /dir3
> hdfs storagepolicies -getStoragePolicy -path /dir3/subdir1
> hdfs storagepolicies -getStoragePolicy -path /dir3/subdir2
> hdfs storagepolicies -getStoragePolicy -path /dir3/subdir3
> ...
> hdfs storagepolicies -getStoragePolicy -path /dirN
> ...
> hdfs storagepolicies -getStoragePolicy -path /dirN/subdirN/subsubdirN
> ...{code}
> In my current environment for example, a policy was configured for /data/blah
> which doesn't show when trying
> {code:java}
> hdfs storagepolicies -getStoragePolicy -path /data{code}
> and I had no way of knowing that I had to do:
> {code:java}
> hdfs storagepolicies -getStoragePolicy -path /data/blah{code}
> other than trial and error of trying every directory and every subdirectory
> in hdfs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
