Hi,
I have a working glusterfs setup running on Centos 5.3 with
glusterfs-2.0.4 (compiled from the source RPM)
fuse-2.7.4-1
dkms-fuse-2.7.4-1.rf
autofs-5.0.1-0.rc2.102
kernel 2.6.18-128.1.10.el5
and this all works just fine - autofs mounts the file system as you would
expect and this has been in production for some time.
However, if I try and upgrade any of the components, it breaks in that the
autofs mount will hang rather than completing the mount.
Mounting the file system by hand with an explicit mount command always works
correctly.
I've tried several versions of glusterfs later than the above including the
latest 3.0.2-1 with exactly the same result.
Additionally keeping that version of gluster and updating any of the other
components also seems to break it, although I've not been able to test all the
combinations - certainly the following set doesn't work either:
glusterfs-3.0.2-1
dkms-fuse-2.7.4-1.nodist.rf
fuse-2.7.4-8.el5
autofs-5.0.1-0.rc2.131.el5_4.1
2.6.18-164.11.1.el5
I wonder if someone on the list can help me, as I've seen nothing in bugzilla
relating to this.
Relevant information follows (for my test rig only)
Server volfile is:
---snip---
[l3ad...@oy-centos-5_3-buildserver glusterfs]$ cat /etc/glusterfs/glusterfsd.vol
## Export volume "images-brick" with the contents of /export/images directory
volume posix
type storage/posix
option directory /export/shared/
end-volume
volume locks
type features/locks
subvolumes posix
end-volume
volume server
type protocol/server
option transport-type tcp/server
subvolumes locks
option auth.addr.locks.allow *
end-volume
---snip---
Client volfile:
---snip---
[l3ad...@oy-centos-5_3-buildserver glusterfs]$ cat /etc/glusterfs/glusterfs.vol
volume oy-centos-5_3-buildserver
type protocol/client
option transport-type tcp/client
option remote-host 127.0.0.1
option remote-subvolume locks
end-volume
---snip---
/etc/auto.master has the following:
---snip---
/mnt/auto /etc/auto.d/auto.gluster --timeout=60 --ghost
---snip---
and auto.gluster has
---snip---
# Mount the glustered file system
shared -fstype=glusterfs :/etc/glusterfs/glusterfs.vol
---snip---
Mounting the gluster file system directly works fine:
---snip---
sudo mount -t glusterfs /etc/glusterfs/glusterfs.vol /mnt/auto/shared/
[l3ad...@oy-centos-5_3-buildserver ~]$ df /mnt/auto/shared
Filesystem 1K-blocks Used Available Use% Mounted on
glusterfs#/etc/glusterfs/glusterfs.vol
2031360 543744 1382656 29% /mnt/auto/shared
---snip---
starting autofs and attempting to access the mounted directory eg
ls /mnt/auto/shared/
causes the glusterfs to hang leaving a process list like this:
[l3ad...@oy-centos-5_3-buildserver glusterfs]$ pstree -p | grep glu
|-automount(22819)-+-mount(22830)---mount.glusterfs(22831)---glusterfs(22880)---glusterfs(22881)
|-glusterfs(22882)---{glusterfs}(22883)
|-glusterfsd(22356)---{glusterfsd}(22357)
Running gdb against 22882 during the hang shows:
(gdb) bt
#0 0x00c81402 in __kernel_vsyscall ()
#1 0x00ee7473 in __xsta...@glibc_2.1 () from /lib/libc.so.6
#2 0x00df66ec in stat64 () from /usr/lib/glusterfs/3.0.0/xlator/mount/fuse.so
#3 0x00df419c in init (this_xl=0x88db1f8) at fuse-bridge.c:3368
#4 0x00b2293d in xlator_init (xl=0x88db1f8) at xlator.c:940
#5 0x00b22583 in xlator_init_rec (xl=0x88db1f8) at xlator.c:833
#6 0x00b226e6 in xlator_tree_init (xl=0x88db1f8) at xlator.c:871
#7 0x0804b299 in _xlator_graph_init ()
#8 0x0804b433 in glusterfs_graph_init ()
#9 0x0804d40c in main ()
(gdb) directory /home/l3admin/rpmbuild/BUILD/glusterfs-3.0.0/glusterfsd/src
Source directories searched:
/home/l3admin/rpmbuild/BUILD/glusterfs-3.0.0/glusterfsd/src:$cdir:$cwd
(gdb) list *0x00df419c
0xdf419c is in init (fuse-bridge.c:3368).
3363 gf_log ("fuse", GF_LOG_ERROR,
3364 "Mandatory option 'mountpoint' is not
specified.");
3365 goto cleanup_exit;
3366 }
3367
3368 if (stat (value_string, &stbuf) != 0) {
3369 if (errno == ENOENT) {
3370 gf_log (this_xl->name, GF_LOG_ERROR,
3371 "%s %s does not exist",
3372 ZR_MOUNTPOINT_OPT, value_string);
(gdb)
(gdb) select-frame 3
(gdb) print value_string
$1 = 0x88da198 "/mnt/auto/shared"
By the time I'd got here, the spawned process on 22883 had died (is there a
watchdog of some sort?) so I repeated the exercise and ran gdb on the watchdog
process (which I think was pid 22881) getting this:
(gdb) bt
#0 0x0013d402 in __kernel_vsyscall ()
#1 0x001cf996 in nanosleep () from /lib/libc.so.6
#2 0x0020915c in usleep () from /lib/libc.so.6
#3 0x00d7fa2d in gf_timer_proc (ctx=0x8560008) at timer.c:177
#4 0x0068573b in start_thread () from /lib/libpthread.so.0
#5 0x0020fcfe in clone () from /lib/libc.so.6
And so I presume that this is waiting for some communication from the process
which spawned it, indicating that the mount was complete???
Regards to all
Phil
--
Director, Layer3 Systems Ltd
Layer3 Systems Limited is registered in England. Company no 3130393
43 Pendle Road, Streatham, London, SW16 6RT
tel: 020 8769 4484
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
