[ https://issues.apache.org/jira/browse/HDFS-15289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094884#comment-17094884 ]
Virajith Jalaparti commented on HDFS-15289: ------------------------------------------- Thanks for your response [~umamaheswararao]. bq. One thought is, admin commands use -fs option and specify the required nn address. Yes, this is what we are leaning towards along with setting {{-Dfs.hdfs.impl=DistributedFileSystem}} when running HAAdmin and DFSAdmin. These configs can work in the short-term before HAAdmin and DFSAdmin are completely moved to {{ViewFSOverLoadScheme}}. bq. If users access DFS directly, they may need to get the childFileSystems from ViewFSOverloadScheme and check the instanceOf. This makes sense for cases where we must use {{DistributedFileSystem}} as a library. [~abhishekd] had to make some changes to {{ViewFileSystem}} to change the scope of some classes. He can post more about the details for this. I suspect some of these were needed due the packaging chosen for the child class and might not be needed if the new class remains in the existing packaging of {{org.apache.hadoop.fs.viewfs}}. > Allow viewfs mounts with hdfs scheme and centralized mount table > ---------------------------------------------------------------- > > Key: HDFS-15289 > URL: https://issues.apache.org/jira/browse/HDFS-15289 > Project: Hadoop HDFS > Issue Type: New Feature > Components: fs > Affects Versions: 3.2.0 > Reporter: Uma Maheswara Rao G > Assignee: Uma Maheswara Rao G > Priority: Major > Attachments: ViewFSOverloadScheme - V1.0.pdf > > > ViewFS provides flexibility to mount different filesystem types with mount > points configuration table. Additionally viewFS provides flexibility to > configure any fs (not only HDFS) scheme in mount table mapping. This approach > is solving the scalability problems, but users need to reconfigure the > filesystem to ViewFS and to its scheme. This will be problematic in the case > of paths persisted in meta stores, ex: Hive. In systems like Hive, it will > store uris in meta store. So, changing the file system scheme will create a > burden to upgrade/recreate meta stores. In our experience many users are not > ready to change that. > Router based federation is another implementation to provide coordinated > mount points for HDFS federation clusters. Even though this provides > flexibility to handle mount points easily, this will not allow > other(non-HDFS) file systems to mount. So, this does not solve the purpose > when users want to mount external(non-HDFS) filesystems. > So, the problem here is: Even though many users want to adapt to the scalable > fs options available, technical challenges of changing schemes (ex: in meta > stores) in deployments are obstructing them. > So, we propose to allow hdfs scheme in ViewFS like client side mount system > and provision user to create mount links without changing URI paths. > I will upload detailed design doc shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org