[
https://issues.apache.org/jira/browse/HADOOP-17060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129736#comment-17129736
]
Uma Maheswara Rao G commented on HADOOP-17060:
--
Hi [~smajeti],
Copied my PR comment here:
I have verified symlinks behavior in HDFS.
{code:java}
public void testSymlinkOnHDFS() throws Exception {
// add ur hdfs uri here: ex hdfs://10.0.1.75:9000
URI hdfsURI = dfs.getUri();
FileSystem.enableSymlinks();
try (FileSystem hdfs = new DistributedFileSystem()) {
hdfs.initialize(hdfsURI, new HdfsConfiguration());
final Path targetLinkDir = new Path("/user", "targetRegularDir");
hdfs.mkdirs(targetLinkDir)
Path symLinkDir = new Path("/links/linkDir");
hdfs.createSymlink(targetLinkDir, symLinkDir, true);
// ListStatus Test
FileStatus[] listStatus = hdfs.listStatus(new Path("/links"));
FileStatus fsFromLs = listStatus[0]; // FileStatus of /links/linkDir
Assert.assertEquals(fsFromLs.isDirectory(), false);
Assert.assertEquals("/links/linkDir",
Path.getPathWithoutSchemeAndAuthority(fsFromLs.getPath()).toString());
// GetFileStatus test
// FileStatus of /links/linkDir
FileStatus fileStatus = hdfs.getFileStatus(symLinkDir);
Assert.assertEquals(true, fileStatus.isDirectory());
// resolved to FileStatus of /user/targetRegularDir
Assert.assertEquals("/user/targetRegularDir", Path
.getPathWithoutSchemeAndAuthority(fileStatus.getPath()).toString());
}
}
{code}
{{}}
It turns out that the behavior of listStatus and GetFileStatus are different.
They both returning different FileStatus. Same behavior in ViewFS also.
GetFileStatus(/test), just runs on resolved path directly. So, it will not be
represented as symLink.
ListStatus(/) of gets /test as children FileStatus object. But that represents
as symLink.
Probably we should just clarify the behavior in user guide and API docs about
the behaviors in symlinks case? Otherwise fixing this needs to be done all
other places and it will be incompatible change across.
One advantage I see with the existing behavior is that, with listStatus we can
know wether dir is symlink. If one wants to know targetFs details, then issue
GetFileStatus on that path will resolved to targetFS and gets the FileStatus at
targetFS.
I will also check with Sanjay on this and update here.
> listStatus and getFileStatus behave inconsistent in the case of ViewFs
> implementation for isDirectory
> -
>
> Key: HADOOP-17060
> URL: https://issues.apache.org/jira/browse/HADOOP-17060
> Project: Hadoop Common
> Issue Type: Bug
> Components: viewfs
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Srinivasu Majeti
>Assignee: Uma Maheswara Rao G
>Priority: Major
> Labels: viewfs
>
> listStatus implementation in ViewFs and getFileStatus does not return
> consistent values for an element on isDirectory value. listStatus returns
> isDirectory of all softlinks as false and getFileStatus returns isDirectory
> as true.
> {code:java}
> [hdfs@c3121-node2 ~]$ /usr/jdk64/jdk1.8.0_112/bin/java -cp `hadoop
> classpath`:./hdfs-append-1.0-SNAPSHOT.jar LauncherGetFileStatus "/"
> FileStatus of viewfs://c3121/testme21may isDirectory:false
> FileStatus of viewfs://c3121/tmp isDirectory:false
> FileStatus of viewfs://c3121/foo isDirectory:false
> FileStatus of viewfs://c3121/tmp21may isDirectory:false
> FileStatus of viewfs://c3121/testme isDirectory:false
> FileStatus of viewfs://c3121/testme2 isDirectory:false <--- returns false
> FileStatus of / isDirectory:true
> [hdfs@c3121-node2 ~]$ /usr/jdk64/jdk1.8.0_112/bin/java -cp `hadoop
> classpath`:./hdfs-append-1.0-SNAPSHOT.jar LauncherGetFileStatus /testme2
> FileStatus of viewfs://c3121/testme2/dist-copynativelibs.sh isDirectory:false
> FileStatus of viewfs://c3121/testme2/newfolder isDirectory:true
> FileStatus of /testme2 isDirectory:true <--- returns true
> [hdfs@c3121-node2 ~]$ {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
