[
https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12572599#action_12572599
]
craigm edited comment on HADOOP-4 at 2/26/08 10:55 AM:
----------------------------------------------------------------
Hi Pete,
The block stuff in fuse is appallingly documented. I have hunted the Web for
info on this all afternoon, to understand it further. To be honest, the only
thing I have found useful is reading the source of ntfs-3g.c at
http://ntfs-3g.cvs.sourceforge.net/ntfs-3g/ntfs-3g/src/ntfs-3g.c?revision=1.106&view=markup
I test I did do a few days ago was to comparing reading an NFS mounted file
directly vs, the same file read via NFS via a FUSE fs -
http://mattwork.potsdam.edu/projects/wiki/index.php/Rofs#rofs_code_.28C.29
(ROFS, the Read-Only Filesystem). Speed results were fairly comparable between
NFS & NFS+ROFS, so it suggests that FUSE doesnt add too much overhead to IO.
Hence then we can only suspect that the problem is in either (a) JNI interface,
or (b) the size of the reads we're performing. A simple C tool can be generated
to exclude (a).
I dont have any objections to pretty large buffer sizes for fuse_dfs.c - HDFS
is designed for large files, and streaming read access.
Btw, you mentioned you are re-exporting the mounted FS as NFS - have you had
any issues vs the issues described in fuses' README.NFS?
Regards
Craig
was (Author: craigm):
Hi Pete,
The block stuff in fuse is appallingly documented. I have hunted the Web for
info on this all afternoon, to understand it further. To be honest, the only
thing I have found useful is reading the source of ntfs-3g.c at
http://ntfs-3g.cvs.sourceforge.net/ntfs-3g/ntfs-3g/src/ntfs-3g.c?revision=1.106&view=markup
I test I did do a few days ago was to comparing reading an NFS mounted file
directly vs, the same file read via NFS via a FUSE fs -
http://mattwork.potsdam.edu/projects/wiki/index.php/Rofs#rofs_code_.28C.29
(ROFS, the Read-Only Filesystem). Speed results were fairly comparable between
NFS & NFS+ROFS, so it suggests that FUSE doesnt add too much overhead to IO.
Hence then we can only suspect that the problem is in either (a) JNI interface,
or (b) the size of the reads we're performing.
I dont have any objections to pretty large buffer sizes for fuse_dfs.c - HDFS
is designed for large files, and streaming read access.
Btw, you mentioned you are re-exporting the mounted FS as NFS - have you had
any issues vs the issues described in fuses' README.NFS?
Regards
Craig
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop Core
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Pete Wyckoff
> Attachments: fuse-dfs.tar.gz, fuse-dfs.tar.gz, fuse-dfs.tar.gz,
> fuse-dfs.tar.gz, fuse-dfs.tar.gz,
> fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz,
> fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz,
> fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c, fuse_dfs.c, fuse_dfs.c, fuse_dfs.c,
> fuse_dfs.sh, Makefile
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
