[
https://issues.apache.org/jira/browse/HDFS-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641309#comment-13641309
]
Brock Noland commented on HDFS-4750:
------------------------------------
Hi Brandon,
Great to see this proposal! I am fine with using a new JIRA for this, but if we
do so, should HDFS-252 be closed as a duplicate? As you know I created an
Apache Licensed NFS4 proxy for HDFS
(https://github.com/cloudera/hdfs-nfs-proxy). I have a couple
questions/comments:
The proposal says that waiting "10 milliseconds" should be able to convert most
writes to sequential writes. I am curious has this been tested under load on
modern kernels? The reason I ask is that I found that often the NFS4 proxy has
to wait much longer than 10 milliseconds to receive the pre-requisite writes.
It's possible that behavior is NFS4 only.
Before implementing the NFS4 proxy I implemented a NFS3 proxy as you propose.
Unfortunately I deleted the git repo when I became frustrated with the mismatch
between NFS3 and HDFS semantics. If I remember correctly, one example was that
when I had a small file, a small append resulted in a write of the entire file.
I cannot remember exactly how it behaved with larger files. Have you
encountered this? If so, how will it be handled?
Another problem I ran into was that since NFS3 doesn't have a close, I was
never sure when to close the HDFS file handle. I see that you plan to handle
this by idle closing file handles. I thought about this approach as well but my
concern was it will often be the case that there is data which has not been
"synced" to HDFS when the native program has closed the file. Therefore there
are races with other clients being able to see that data. I am not 100% up the
latest of when a file length is updated in HDFS, but I believe there is a
similar issue with the length metadata as well. How will this be handled?
Once again, great work on the proposal!
Cheers,
Brock
> Support NFSv3 interface to HDFS
> -------------------------------
>
> Key: HDFS-4750
> URL: https://issues.apache.org/jira/browse/HDFS-4750
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 3.0.0
> Reporter: Brandon Li
> Assignee: Brandon Li
> Attachments: HADOOP-NFS-Proposal.pdf
>
>
> Access HDFS is usually done through HDFS Client or webHDFS. Lack of seamless
> integration with client’s file system makes it difficult for users and
> impossible for some applications to access HDFS. NFS interface support is one
> way for HDFS to support such easy integration.
> This JIRA is to track the NFS protocol support for accessing HDFS. With HDFS
> client, webHDFS and the NFS interface, HDFS will be easier to access and be
> able support more applications and use cases.
> We will upload the design document and the initial implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
