[
https://issues.apache.org/jira/browse/HDFS-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota updated HDFS-13683:
------------------------------
Description:
In a federated cluster (defaultFS = viewfs://cluster3), getStoragePolicy with
hdfs uri will run into the following error:
{noformat}
[root@cdh6-0-0-centos73-17443-1 ~]# hdfs storagepolicies -getStoragePolicy
-path
hdfs://ns2/hbase/WALs/cdh6-0-0-centos73-17443-4.vpc.cloudera.com,22101,1527272361896
IllegalArgumentException: Wrong FS:
hdfs://ns2/hbase/WALs/cdh6-0-0-centos73-17443-4.vpc.cloudera.com,22101,1527272361896,
expected: viewfs://cluster3/
{noformat}
Taking a quick look at the code, I think [this|] is the culprit:
{code}
Path p = new Path(path);
final FileSystem fs = FileSystem.get(conf);
// should do: final FileSystem fs = p.getFilesystem(conf);
{code}
We should have a review of all shell and see if anything else is missing. At
the minimum, we should fix all places in StoragePolicyAdmin.
was:
In a federated cluster (defaultFS = viewfs://cluster3), getStoragePolicy with
hdfs uri will run into the following error:
{noformat}
[root@cdh6-0-0-centos73-17443-1 ~]# hdfs storagepolicies -getStoragePolicy
-path
hdfs://ns2/hbase/WALs/cdh6-0-0-centos73-17443-4.vpc.cloudera.com,22101,1527272361896
IllegalArgumentException: Wrong FS:
hdfs://ns2/hbase/WALs/cdh6-0-0-centos73-17443-4.vpc.cloudera.com,22101,1527272361896,
expected: viewfs://cluster3/
{noformat}
Taking a quick look at the code, I think
[this|https://github.com/apache/hadoop/blob/30941d99c9014431981eeb09ab24e90bef629fee/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java#L106]
is the culprit:
{code}
Path p = new Path(path);
final FileSystem fs = FileSystem.get(conf);
// should do: final FileSystem fs = p.getFilesystem(conf);
{code}
We should have a review of all shell and see if anything else is missing. At
the minimum, we should fix all places in StoragePolicyAdmin.
> HDFS StoragePolicy commands should work with Federation
> -------------------------------------------------------
>
> Key: HDFS-13683
> URL: https://issues.apache.org/jira/browse/HDFS-13683
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, tools
> Affects Versions: 3.0.0
> Reporter: Gabor Bota
> Assignee: Gabor Bota
> Priority: Major
>
> In a federated cluster (defaultFS = viewfs://cluster3), getStoragePolicy with
> hdfs uri will run into the following error:
> {noformat}
> [root@cdh6-0-0-centos73-17443-1 ~]# hdfs storagepolicies -getStoragePolicy
> -path
> hdfs://ns2/hbase/WALs/cdh6-0-0-centos73-17443-4.vpc.cloudera.com,22101,1527272361896
> IllegalArgumentException: Wrong FS:
> hdfs://ns2/hbase/WALs/cdh6-0-0-centos73-17443-4.vpc.cloudera.com,22101,1527272361896,
> expected: viewfs://cluster3/
> {noformat}
> Taking a quick look at the code, I think [this|] is the culprit:
> {code}
> Path p = new Path(path);
> final FileSystem fs = FileSystem.get(conf);
> // should do: final FileSystem fs = p.getFilesystem(conf);
> {code}
> We should have a review of all shell and see if anything else is missing. At
> the minimum, we should fix all places in StoragePolicyAdmin.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
