[
https://issues.apache.org/jira/browse/HADOOP-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15939527#comment-15939527
]
Andrew Wang commented on HADOOP-14210:
--------------------------------------
I don't have a fully educated opinion on this topic, but I normally look to
Unixy behavior to determine user expectations of HDFS. "ls -R /" lists
everything. When I stat a mount point, it's a directory, not a symlink.
Symlinks are currently not enabled in HDFS, so no downstream apps have ever
encountered a symlink. Again looking at Unix as a guide, applications still
don't handle symlinks correctly from a correctness or security POV after
decades of existence. So, if we can roll out VFS without requiring apps to
understand symlinks, I'd be happy.
A lot of our apps use HDFS APIs like the shell or {{FileSystem#listFiles}}.
We'd want these to work the same way regardless of how the VFS is sharded, for
transparency.
Can we dredge up any of the initial motivations from the VFS design docs or
JIRA?
Given that existing FSs can already span to 100s of millions of files, clients
need to handle high scale no matter what. For the CLI ls command, I'm hoping
we're using the iterator-based listing methods, as well as providing a mode
that doesn't try to align columns (which requires buffering all the output).
> Directories are not listed recursively when fs.defaultFs is viewFs
> ------------------------------------------------------------------
>
> Key: HADOOP-14210
> URL: https://issues.apache.org/jira/browse/HADOOP-14210
> Project: Hadoop Common
> Issue Type: Bug
> Components: viewfs
> Affects Versions: 2.7.0
> Reporter: Ajith S
> Labels: viewfs
> Attachments: HDFS-8413.patch
>
>
> Mount a cluster on client throught viewFs mount table
> Example:
> {quote}
> <property>
> <name>fs.defaultFS</name>
> <value>viewfs:///</value>
> </property>
> <property>
> <name>fs.viewfs.mounttable.default.link./nn1</name>
> <value>hdfs://ns1/</value> <!-- HA nameservice -->
> </property>
> <property>
> <name>fs.viewfs.mounttable.default.link./user</name>
> <value>hdfs://host-72:8020/</value>
> </property>
> <property>
> {quote}
> Try to list the files recursively *(hdfs dfs -ls -R / or hadoop fs -ls -R /)*
> only the parent folders are listed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
