Yes, You are right. Thanks Brian. I changed my code back to the original, and used "./fuse_dfs_wrapper.sh dfs://lwm1:54310 /mnt/dfs -d" to mount. It works.
Best regards Weiming On Tue, Dec 22, 2009 at 10:46 PM, Brian Bockelman <[email protected]> wrote: > No. > > The strings *must match* in both hadoop-site.xml and your fuse_dfs arguments. > > For example, in my /etc/hosts, I have: > > [br...@red ~]$ cat /etc/hosts | grep hadoop-name > 172.16.100.8 hadoop-name > > In hadoop-site.xml, I have: > > <property> > <name>fs.default.name</name> > <value>hdfs://hadoop-name:9000</value> > </property> > > If I then did this: > > /usr/bin/fuse_dfs /mnt/hadoop -o > rw,server=172.16.100.8,port=9000,rdbuffer=32768,allow_other > > this would successfully mount, but fail with all sorts of cryptic errors when > I try to use it. I must instead do this: > > /usr/bin/fuse_dfs /mnt/hadoop -o > rw,server=hadoop-name,port=9000,rdbuffer=32768,allow_other > > Your case might be similar. > > Brian > > On Dec 22, 2009, at 8:39 AM, Weiming Lu wrote: > >> Hi, Brian, >> I have added "10.15.62.4 lwm1" in /etc/hosts. >> So, it may be not the reason? >> >> >> Best regard, >> Weiming Lu >> >> On Tue, Dec 22, 2009 at 9:05 PM, Brian Bockelman <[email protected]> >> wrote: >>> Hey Weiming, >>> >>> One "funky" thing about FUSE-DFS: >>> >>> Your host name in hadoop-site.xml and the hostname you use to mount >>> fuse-dfs must be *exactly* the same string. >>> >>> I see you refer to this as lwm1 in hadoop-site.xml and 10.15.62.4 in your >>> FUSE output. This will cause all your issues. >>> >>> I'd advise to change your code back to the original one and make sure >>> everything matches. >>> >>> Brian >>> >>> On Dec 22, 2009, at 5:01 AM, Weiming Lu wrote: >>> >>>> Hi, Eli, Thanks very much. >>>> I added the following code in function dfs_readdir in >>>> src/contrib/fuse-dfs/src/fuse_dfs.c: >>>> fprintf(stderr,"numEntries: %d\n",numEntries); >>>> I can print it correctly. >>>> >>>> Finally, I found the following code in function dfs_readdir in >>>> fuse_dfs.c maybe errous. >>>> const char *const str = info[i].mName + dfs->dfs_uri_len + path_len + >>>> ((path_len == 1 && *path == '/') ? 0 : 1); >>>> >>>> so I modified it as: const char* const str = info[i].mName; >>>> >>>> and then, I recompiled and remount ... >>>> when I "ls /mnt/dfs", the output is : >>>> l...@lwm10:~$ ls /mnt/dfs >>>> ls: cannot access /mnt/dfs/hdfs://lwm1:54310/hbase: No such file or >>>> directory >>>> ls: cannot access /mnt/dfs/hdfs://lwm1:54310/home: No such file or >>>> directory >>>> ls: cannot access /mnt/dfs/hdfs://lwm1:54310/movies: No such file or >>>> directory >>>> ls: cannot access /mnt/dfs/hdfs://lwm1:54310/system: No such file or >>>> directory >>>> ls: cannot access /mnt/dfs/hdfs://lwm1:54310/user: No such file or >>>> directory >>>> hdfs://lwm1:54310/hbase hdfs://lwm1:54310/movies hdfs://lwm1:54310/user >>>> hdfs://lwm1:54310/home hdfs://lwm1:54310/system >>>> >>>> >>>> Fortunately, we can access the subdir by "cd /mnt/dfs/user". And, >>>> "cannot access /mnt/dfs/hdfs://lwm1:......" maybe caused by my >>>> modification. >>>> >>>> The debug message is: >>>> port=54310,server=10.15.62.4 >>>> fuse-dfs didn't recognize /mnt/dfs,-2 >>>> fuse-dfs ignoring option -d >>>> FUSE library version: 2.8.1 >>>> nullpath_ok: 0 >>>> unique: 1, opcode: INIT (26), nodeid: 0, insize: 56 >>>> INIT: 7.9 >>>> flags=0x0000000b >>>> max_readahead=0x00020000 >>>> INIT: 7.12 >>>> flags=0x00000011 >>>> max_readahead=0x00020000 >>>> max_write=0x00020000 >>>> unique: 1, success, outsize: 40 >>>> unique: 2, opcode: GETATTR (3), nodeid: 1, insize: 56 >>>> getattr / >>>> fuse_dfs TRACE - getattr / >>>> unique: 2, success, outsize: 120 >>>> unique: 3, opcode: OPENDIR (27), nodeid: 1, insize: 48 >>>> unique: 3, success, outsize: 32 >>>> unique: 4, opcode: READDIR (28), nodeid: 1, insize: 80 >>>> readdir[0] from 0 >>>> fuse_dfs TRACE - readdir / >>>> numEntries: 5 >>>> info[0].mName = hdfs://lwm1:54310/hbase >>>> dfs->dfs_uri_len = 23 >>>> path_len = 1 >>>> stat string: hdfs://cadal:54310/hbase >>>> info[1].mName = hdfs://lwm1:54310/home >>>> dfs->dfs_uri_len = 23 >>>> path_len = 1 >>>> stat string: hdfs://cadal:54310/home >>>> info[2].mName = hdfs://lwm1:54310/movies >>>> dfs->dfs_uri_len = 23 >>>> path_len = 1 >>>> stat string: hdfs://cadal:54310/movies >>>> info[3].mName = hdfs://lwm1:54310/system >>>> dfs->dfs_uri_len = 23 >>>> path_len = 1 >>>> stat string: hdfs://cadal:54310/system >>>> info[4].mName = hdfs://lwm1:54310/user >>>> dfs->dfs_uri_len = 23 >>>> path_len = 1 >>>> stat string: hdfs://lwm1:54310/user >>>> unique: 4, success, outsize: 336 >>>> unique: 5, opcode: GETATTR (3), nodeid: 1, insize: 56 >>>> getattr / >>>> fuse_dfs TRACE - getattr / >>>> unique: 5, success, outsize: 120 >>>> unique: 6, opcode: LOOKUP (1), nodeid: 1, insize: 46 >>>> LOOKUP /hdfs: >>>> getattr /hdfs: >>>> fuse_dfs TRACE - getattr /hdfs: >>>> Exception in thread "main" java.lang.IllegalArgumentException: >>>> Pathname /hdfs: from /hdfs: is not a valid DFS filename. >>>> at >>>> org.apache.hadoop.dfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:125) >>>> at >>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:390) >>>> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:667) >>>> Call to org.apache.hadoop.fs.FileSystem::exists failed! >>>> unique: 6, error: -2 (No such file or directory), outsize: 16 >>>> unique: 7, opcode: LOOKUP (1), nodeid: 1, insize: 46 >>>> LOOKUP /hdfs: >>>> getattr /hdfs: >>>> fuse_dfs TRACE - getattr /hdfs: >>>> Exception in thread "Thread-7" java.lang.IllegalArgumentException: >>>> Pathname /hdfs: from /hdfs: is not a valid DFS filename. >>>> at >>>> org.apache.hadoop.dfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:125) >>>> at >>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:390) >>>> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:667) >>>> Call to org.apache.hadoop.fs.FileSystem::exists failed! >>>> unique: 7, error: -2 (No such file or directory), outsize: 16 >>>> unique: 8, opcode: LOOKUP (1), nodeid: 1, insize: 46 >>>> LOOKUP /hdfs: >>>> getattr /hdfs: >>>> fuse_dfs TRACE - getattr /hdfs: >>>> Exception in thread "Thread-8" java.lang.IllegalArgumentException: >>>> Pathname /hdfs: from /hdfs: is not a valid DFS filename. >>>> at >>>> org.apache.hadoop.dfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:125) >>>> at >>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:390) >>>> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:667) >>>> Call to org.apache.hadoop.fs.FileSystem::exists failed! >>>> unique: 8, error: -2 (No such file or directory), outsize: 16 >>>> unique: 9, opcode: LOOKUP (1), nodeid: 1, insize: 46 >>>> LOOKUP /hdfs: >>>> getattr /hdfs: >>>> fuse_dfs TRACE - getattr /hdfs: >>>> Exception in thread "main" java.lang.IllegalArgumentException: >>>> Pathname /hdfs: from /hdfs: is not a valid DFS filename. >>>> at >>>> org.apache.hadoop.dfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:125) >>>> at >>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:390) >>>> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:667) >>>> Call to org.apache.hadoop.fs.FileSystem::exists failed! >>>> unique: 9, error: -2 (No such file or directory), outsize: 16 >>>> unique: 10, opcode: LOOKUP (1), nodeid: 1, insize: 46 >>>> LOOKUP /hdfs: >>>> getattr /hdfs: >>>> fuse_dfs TRACE - getattr /hdfs: >>>> Exception in thread "Thread-8" java.lang.IllegalArgumentException: >>>> Pathname /hdfs: from /hdfs: is not a valid DFS filename. >>>> at >>>> org.apache.hadoop.dfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:125) >>>> at >>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:390) >>>> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:667) >>>> Call to org.apache.hadoop.fs.FileSystem::exists failed! >>>> unique: 10, error: -2 (No such file or directory), outsize: 16 >>>> unique: 11, opcode: READDIR (28), nodeid: 1, insize: 80 >>>> unique: 11, success, outsize: 16 >>>> unique: 12, opcode: RELEASEDIR (29), nodeid: 1, insize: 64 >>>> unique: 12, success, outsize: 16 >>>> >>>> >>>> >>>> >>>> On Mon, Dec 21, 2009 at 12:53 PM, Eli Collins <[email protected]> wrote: >>>>> By the way, feel free to send your response to just me >>>>> ([email protected]) if you don't want to spam the list. >>>>> >>>>> Even if you use the list if you also include my email I'll see your >>>>> responses faster. >>>>> >>>>> Thanks, >>>>> Eli >>>>> >>>>> >>>>> On Sun, Dec 20, 2009 at 8:51 PM, Eli Collins <[email protected]> wrote: >>>>>>> fuse_dfs TRACE - readdir / >>>>>>> unique: 4, success, outsize: 200 >>>>>>> unique: 5, opcode: RELEASEDIR (29), nodeid: 1, insize: 64 >>>>>>> unique: 5, success, outsize: 16 >>>>>>> >>>>>>> Does It seem OK? >>>>>> >>>>>> Hm, seems like it's not finding any directory entries. Mind putting a >>>>>> printf in dfs_readdir after hdfsListDirectories in >>>>>> fuse_impls_readdir.c to see what numEntries is? >>>>>> >>>>>> Thanks, >>>>>> Eli >>>>>> >>>>> >>> >>> > >
