[
https://issues.apache.org/jira/browse/HDFS-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16177689#comment-16177689
]
Mukul Kumar Singh commented on HDFS-11968:
------------------------------------------
Hi [~arpitagarwal], I ran the following commands on a viewfs,hdfs and webhdfs
based instances and following are the list of commands and their output.
ViewFS based uri
{code}
*core-site.xml*
<property>
<name>fs.default.name</name>
<value>viewfs://cluster1</value>
</description>
<final>true</final>
</property>
<property>
<name>fs.viewfs.mounttable.cluster1.link./home</name>
<value>hdfs://127.0.0.1:8020/dir1</value>
</property>
<property>
<name>fs.viewfs.mounttable.cluster1.link./tmp</name>
<value>hdfs://127.0.0.1:8020/dir2</value>
</property>
*list storage policies*
[root@304a057b260f opt]# /opt/hadoop-3.1.0-SNAPSHOT/bin/hdfs storagepolicies
-listPolicies
Block Storage Policies:
BlockStoragePolicy{COLD:2, storageTypes=[ARCHIVE],
creationFallbacks=[], replicationFallbacks=[]}
BlockStoragePolicy{WARM:5, storageTypes=[DISK, ARCHIVE],
creationFallbacks=[DISK, ARCHIVE], replicationFallbacks=[DISK, ARCHIVE]}
BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[],
replicationFallbacks=[ARCHIVE]}
BlockStoragePolicy{ONE_SSD:10, storageTypes=[SSD, DISK],
creationFallbacks=[SSD, DISK], replicationFallbacks=[SSD, DISK]}
BlockStoragePolicy{ALL_SSD:12, storageTypes=[SSD],
creationFallbacks=[DISK], replicationFallbacks=[DISK]}
BlockStoragePolicy{LAZY_PERSIST:15, storageTypes=[RAM_DISK, DISK],
creationFallbacks=[DISK], replicationFallbacks=[DISK]}
*set storage policy*
[root@304a057b260f opt]# /opt/hadoop-3.1.0-SNAPSHOT/bin/hdfs storagepolicies
-setStoragePolicy -path /home -policy ALL_SSD
Set storage policy ALL_SSD on /home
*get storage policies after setting a policy*
[root@304a057b260f opt]# /opt/hadoop-3.1.0-SNAPSHOT/bin/hdfs storagepolicies
-getStoragePolicy -path /home
The storage policy of /home:
BlockStoragePolicy{ALL_SSD:12, storageTypes=[SSD], creationFallbacks=[DISK],
replicationFallbacks=[DISK]
*unset storage policy*
[root@304a057b260f opt]# /opt/hadoop-3.1.0-SNAPSHOT/bin/hdfs storagepolicies
-unsetStoragePolicy -path /home
Unset storage policy from /home
{code}
Web HDFS based uri
{code}
*core site*
<name>fs.default.name</name>
<value>webhdfs://127.0.0.1:9870/</value>
*list storage policy*
[root@304a057b260f opt]# /opt/hadoop-3.1.0-SNAPSHOT/bin/hdfs storagepolicies
-listPolicies
Block Storage Policies:
BlockStoragePolicy{COLD:2, storageTypes=[ARCHIVE],
creationFallbacks=[], replicationFallbacks=[]}
BlockStoragePolicy{WARM:5, storageTypes=[DISK, ARCHIVE],
creationFallbacks=[DISK, ARCHIVE], replicationFallbacks=[DISK, ARCHIVE]}
BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[],
replicationFallbacks=[ARCHIVE]}
BlockStoragePolicy{ONE_SSD:10, storageTypes=[SSD, DISK],
creationFallbacks=[SSD, DISK], replicationFallbacks=[SSD, DISK]}
BlockStoragePolicy{ALL_SSD:12, storageTypes=[SSD],
creationFallbacks=[DISK], replicationFallbacks=[DISK]}
BlockStoragePolicy{LAZY_PERSIST:15, storageTypes=[RAM_DISK, DISK],
creationFallbacks=[DISK], replicationFallbacks=[DISK]}
*set storage policy*
[root@304a057b260f opt]# /opt/hadoop-3.1.0-SNAPSHOT/bin/hdfs storagepolicies
-setStoragePolicy -path /dir2 -policy WARM
Set storage policy WARM on /dir2
*get storage policy*
[root@304a057b260f opt]# /opt/hadoop-3.1.0-SNAPSHOT/bin/hdfs storagepolicies
-getStoragePolicy -path /dir2
The storage policy of /dir2:
BlockStoragePolicy{WARM:5, storageTypes=[DISK, ARCHIVE],
creationFallbacks=[DISK, ARCHIVE], replicationFallbacks=[DISK, ARCHIVE]}
{code}
HDFS based uri
{code}
*core site*
<name>fs.default.name</name>
<value>hdfs://127.0.0.1:8020/</value>
*list storage policy*
[root@304a057b260f opt]# /opt/hadoop-3.1.0-SNAPSHOT/bin/hdfs storagepolicies
-listPolicies
Block Storage Policies:
BlockStoragePolicy{COLD:2, storageTypes=[ARCHIVE],
creationFallbacks=[], replicationFallbacks=[]}
BlockStoragePolicy{WARM:5, storageTypes=[DISK, ARCHIVE],
creationFallbacks=[DISK, ARCHIVE], replicationFallbacks=[DISK, ARCHIVE]}
BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[],
replicationFallbacks=[ARCHIVE]}
BlockStoragePolicy{ONE_SSD:10, storageTypes=[SSD, DISK],
creationFallbacks=[SSD, DISK], replicationFallbacks=[SSD, DISK]}
BlockStoragePolicy{ALL_SSD:12, storageTypes=[SSD],
creationFallbacks=[DISK], replicationFallbacks=[DISK]}
BlockStoragePolicy{LAZY_PERSIST:15, storageTypes=[RAM_DISK, DISK],
creationFallbacks=[DISK], replicationFallbacks=[DISK]}
*set storage policy*
[root@304a057b260f opt]# /opt/hadoop-3.1.0-SNAPSHOT/bin/hdfs storagepolicies
-setStoragePolicy -path /dir1 -policy COLD
Set storage policy COLD on /dir1
*get storage policy*
[root@304a057b260f opt]# /opt/hadoop-3.1.0-SNAPSHOT/bin/hdfs storagepolicies
-getStoragePolicy -path /dir1
The storage policy of /dir1:
BlockStoragePolicy{COLD:2, storageTypes=[ARCHIVE], creationFallbacks=[],
replicationFallbacks=[]}
{code}
> ViewFS: StoragePolicies commands fail with HDFS federation
> ----------------------------------------------------------
>
> Key: HDFS-11968
> URL: https://issues.apache.org/jira/browse/HDFS-11968
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
> Affects Versions: 2.7.1
> Reporter: Mukul Kumar Singh
> Assignee: Mukul Kumar Singh
> Attachments: HDFS-11968.001.patch, HDFS-11968.002.patch,
> HDFS-11968.003.patch, HDFS-11968.004.patch, HDFS-11968.005.patch,
> HDFS-11968.006.patch, HDFS-11968.007.patch, HDFS-11968.008.patch,
> HDFS-11968.009.patch, HDFS-11968.010.patch
>
>
> hdfs storagepolicies command fails with HDFS federation.
> For storage policies commands, a given user path should be resolved to a HDFS
> path and
> storage policy command should be applied onto the resolved HDFS path.
> {code}
> static DistributedFileSystem getDFS(Configuration conf)
> throws IOException {
> FileSystem fs = FileSystem.get(conf);
> if (!(fs instanceof DistributedFileSystem)) {
> throw new IllegalArgumentException("FileSystem " + fs.getUri() +
> " is not an HDFS file system");
> }
> return (DistributedFileSystem)fs;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
