[
https://issues.apache.org/jira/browse/HADOOP-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396260#comment-16396260
]
Hanisha Koneru commented on HADOOP-14210:
-----------------------------------------
It would be good to have {{-ls -R}} operation on ViewFs to work as it would on
Unix. But I do have a concern/ question about recursively listing files/
directories in ViewFs. How are we handling the scenario where one mount target
is a parent of another mount target. For example, in the config below, if we
recursively list the files/ directories under viewFs root, then the files/
directories under {{/user}} will be listed twice (once for {{/nn1}} and once
for {{/user}}). I think this would be a bad experience for users.
{code:java}
fs.defaultFS = viewfs:///
fs.viewfs.mounttable.default.link./nn1 = hdfs://ns1/
fs.viewfs.mounttable.default.link./user = hdfs://ns1/user/
{code}
One option is to duplicate the behavior as is for symlinks in Unix. In Unix,
{{ls -R}} does not list the contents of a symlink's target. We need to add
"*{{-L | --dereference}}*" option to recursively list contents of symlinks
along with directories.
We can copy this behaviour in ViewFs. That is, we recursively list the
contents of mount's target filesystem only when {{-ls -R}} is called with the
option {{-L}}. This would still list the contents of \{{/user}} twice for the
scenario mentioned above, but I think that should be fine.
Would love to hear thoughts on this.
> Directories are not listed recursively when fs.defaultFs is viewFs
> ------------------------------------------------------------------
>
> Key: HADOOP-14210
> URL: https://issues.apache.org/jira/browse/HADOOP-14210
> Project: Hadoop Common
> Issue Type: Bug
> Components: viewfs
> Affects Versions: 2.7.0
> Reporter: Ajith S
> Priority: Major
> Labels: viewfs
> Attachments: HDFS-8413.patch
>
>
> Mount a cluster on client throught viewFs mount table
> Example:
> {quote}
> <property>
> <name>fs.defaultFS</name>
> <value>viewfs:///</value>
> </property>
> <property>
> <name>fs.viewfs.mounttable.default.link./nn1</name>
> <value>hdfs://ns1/</value> <!-- HA nameservice -->
> </property>
> <property>
> <name>fs.viewfs.mounttable.default.link./user</name>
> <value>hdfs://host-72:8020/</value>
> </property>
> <property>
> {quote}
> Try to list the files recursively *(hdfs dfs -ls -R / or hadoop fs -ls -R /)*
> only the parent folders are listed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
