[
https://issues.apache.org/jira/browse/HDFS-11575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052022#comment-16052022
]
Mukul Kumar Singh commented on HDFS-11575:
------------------------------------------
Thanks for the review [~jnp], Please find my replies inline.
Can we avoid calling fs.resolvePath(..)? This makes a server call and needs
HDFS to be up when NFS is being deployed. Not too bad, but decoupling would be
better, as it has been without this patch. One possibility is to make path
qualified using file system uri. If creating a URI is lightweight, we may not
need to store path to URI mapping. The mount function already makes a call to
Namenode, URI creation could also be moved there.
bq. This step is needed because entries in the mountable map needs to be parsed
an resolved. This is will perform the step of parsing the
{{InodeTree.ResolveResult}}.
To construct DFSClient, we could use this constructor DFSClient(URI
nameNodeUri, Configuration conf), will save a step. The constructor should take
care of HA case.
bq. Done
I am wondering why do we need 'hostMapAddress', since we are caching DFSClient
itself, the lookups to this map would be rare, in that case a DNS resolution is
not that bad.
bq. True, access to this map will be rare, however this map is needed when a
cached DFSClient is evicted from the cache. This map will be used to map a name
node id to its corresponding URI.
{{ FileSystem[] childFileSystems = fs.getChildFileSystems()}} This line will
get the children file systems of the target as well. It might be better to get
the target file systems from the mount points. We may not need it altogether if
hostMapAddress is removed.
bq. Great suggestion, done :)
TestViewfsWithNfs3: Please add a test for rename to check for unsupported error.
bq. Done
Add a test to start NFS service with viewfs over a non-hdfs file system. It is
ok to add it in a follow up jira.
bq. Will add this as part of follow up jira
> Supporting HDFS NFS gateway with Federated HDFS
> -----------------------------------------------
>
> Key: HDFS-11575
> URL: https://issues.apache.org/jira/browse/HDFS-11575
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: nfs
> Reporter: Mukul Kumar Singh
> Assignee: Mukul Kumar Singh
> Attachments: HDFS-11575.001.patch, HDFS-11575.002.patch,
> HDFS-11575.003.patch, HDFS-11575.004.patch, HDFS-11575.005.patch,
> SupportingNFSwithFederatedHDFS.pdf
>
>
> Currently HDFS NFS gateway only supports HDFS as the underlying filesystem.
> Federated HDFS with ViewFS helps in improving the scalability of the name
> nodes. However NFS is not supported with ViewFS.
> With this change, ViewFS using HDFS as the underlying filesystem can be
> exported using NFS. ViewFS mount table will be used to determine the exports
> which needs to be supported.
> Some important points
> 1) This patch only supports HDFS as the underlying filesystem for ViewFS.
> 2) This patch add support to add more than one export point in the NFS gateway
> 3) Root filesystem of the ViewFS will not be mountable for NFS gateway with
> ViewFS,
> however this will not be the case for NFS gateway with HDFS
> 4) A filehandle now apart from the field will also contain an identifier to
> identify the name node, this will be used to map to correct name node for
> file operations.
> Please find the attached pdf document which helps in explaining the design
> and the solution.
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
