On 01/28/2011 12:49 PM, Steve Wilson wrote:
I'm running a pair of replicated/distributed GlusterFS 3.1.2 servers,
each with 8 bricks. Here's the command I used to create the data volume:
gluster volume create post replica 2 transport tcp
pablo:/gluster/01/brick stanley:/gluster/01/brick \
pablo:/gluster/02/brick stanley:/gluster/02/brick \
pablo:/gluster/03/brick stanley:/gluster/03/brick \
pablo:/gluster/04/brick stanley:/gluster/04/brick \
pablo:/gluster/05/brick stanley:/gluster/05/brick \
pablo:/gluster/06/brick stanley:/gluster/06/brick \
pablo:/gluster/07/brick stanley:/gluster/07/brick \
pablo:/gluster/08/brick stanley:/gluster/08/brick
The clients mount the volume using the gluster FUSE client, again
version 3.1.2. Since GlusterFS doesn't yet support exporting
sub-directories directly (it does now for NFS, though, I believe), I
mount the volume as:
/net/.post-volume
and then bind mount a couple of lower sub-directories as:
/net/post (bound to /net/.post-volume/home)
/net/post-scratch (bound to /net/.post-volume/scratch)
I am using this volume for user home directories.
That's the background information... now for the problem. There are
certain files that are almost guaranteed to go missing after less than
a few hours of a user starting up a login session. We use Ubuntu with
GDM as a login greeter and both KDE and Gnome desktops. Here are a
few of the files that usually go missing:
ls: .dmrc: No such file or directory
ls: .pine-debug2: Invalid argument
ls: .xsession-errors.old: No such file or directory
ls: .recently-used.xbel: No such file or directory
Taking .dmrc, for example, I can stop the volume, then remove all
occurrences of this file on all the bricks on both servers. Then
start up the volume again, start the user session again, and after a
period of time (I'm not sure how long), the file comes up missing
again. When the file is missing, this is what it looks like on the
bricks themselves:
server 1:
root@stanley:/gluster# ls -l ./07/brick/home/stevew/.dmrc
---------T 1 stevew sysmgr 0 2011-01-28 08:24
./07/brick/home/stevew/.dmrc
root@stanley:/gluster# ls -l ./02/brick/home/stevew/.dmrc
-rw-r--r-- 1 stevew sysmgr 25 2011-01-28 08:25
./02/brick/home/stevew/.dmrc
server 2:
root@pablo:/gluster# ls -l ./07/brick/home/stevew/.dmrc
---------T 1 stevew sysmgr 0 2011-01-28 08:24
./07/brick/home/stevew/.dmrc
root@pablo:/gluster# ls -l ./02/brick/home/stevew/.dmrc
-rw-r--r-- 1 stevew sysmgr 25 2011-01-28 08:25
./02/brick/home/stevew/.dmrc
Pertinent log messages are:
server 1:
net-.post-volume.log:[2011-01-28 08:16:28.899544] W
[fuse-bridge.c:2432:fuse_xattr_cbk] glusterfs-fuse: 6318974:
GETXATTR() /home/stevew/.dmrc => -1 (Invalid argument)
server 2:
net-.post-volume.log:[2011-01-28 08:10:58.74848] W
[fuse-bridge.c:2432:fuse_xattr_cbk] glusterfs-fuse: 2428788:
GETXATTR() /home/stevew/.dmrc => -1 (No such file or directory)
Any thoughts or ideas?
Thanks!
Steve
Yesterday I disabled the performance.stat-prefetch volume option and
this has improved the situation greatly. I can now do a "ls -alR >
/dev/null" and get no errors on the mounted volume.
Steve
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
