There are a couple of things we can do:
-> Mail us the Glusterfs log files from the NFS server and the glusterfs
servers when the lock script fails. Do file a bug if you can.
-> On the NFS client machine, before you run the mount command, make
sure you run the following command.
$ rpc.statd
-> Run the same perl script but this time at the nfs server over the
glusterfs mount point, not at the NFS client. If it runs fine, it is
probably related to locking over NFS and we'll look at other places to
figure it out.
-Shehjar
Steinmetz, Ian wrote:
I'm seeing an issue where I can't lock files on a NFS exported
GlusterFS mount. I have two servers connected to each other doing AFR
to provide a high available NFS server (mirror the content, one VIP for
NFS mounts to clients). Both of the servers have mounted
"/mnt/glusterfs" using GlusterFS with the client pointing to both
servers. I then export the filesyste with NFS. I grabbed a quick perl
program that tries to lock a file for testing, which fails only on the
glusterfs. When I export a normal directory "/mnt/test" the locking
works.
Any ideas appreciated. I have a feeling I've implemented the
posix/locks option incorrectly.
Both servers are running Ubuntu with identical setups, below are
relevant configs.
r...@gluster01:/mnt/glusterfs# uname -a
Linux gluster01 2.6.31-20-generic-pae #58-Ubuntu SMP Fri Mar 12 06:25:51
UTC 2010 i686 GNU/Linux
r...@gluster01:/mnt/glusterfs# cat /etc/exports
/mnt/glusterfs <ip removed for
security>/25(rw,no_root_squash,no_all_squash,no_subtree_check,sync,insec
ure,fsid=10)
/mnt/test <ip removed for
security>/25(rw,no_root_squash,no_all_squash,no_subtree_check,sync,insec
ure,fsid=11)
* I've tried async, rsync, removing all options except FSID.
r...@gluster02:/etc/glusterfs# cat glusterfs.vol
volume brick1
type protocol/client
option transport-type tcp/client
option remote-host <ip removed for security> # IP address of the remote
brick
option remote-subvolume brick # name of the remote volume
end-volume
volume brick2
type protocol/client
option transport-type tcp/client
option remote-host <ip removed for security> # IP address of the
remote brick
option remote-subvolume brick # name of the remote volume
end-volume
volume afr1
type cluster/afr
subvolumes brick1 brick2
end-volume
volume writebehind
type performance/write-behind
option window-size 4MB
subvolumes afr1
end-volume
volume cache
type performance/io-cache
option cache-size 512MB
subvolumes writebehind
end-volume
volume readahead
type performance/read-ahead
option page-size 128KB # unit in bytes
subvolumes cache
end-volume
volume iothreads
type performance/io-threads
option thread-count 4
option cache-size 64MB
subvolumes readahead
end-volume
r...@gluster02:/etc/glusterfs# cat glusterfsd.vol
volume posix
type storage/posix
option directory /data/export
end-volume
volume locks
type features/posix-locks
option manditory on # tried with and without this, found in a search
of earlier post
subvolumes posix
end-volume
volume brick
type performance/io-threads
option thread-count 8
subvolumes locks
end-volume
volume server
type protocol/server
option transport-type tcp
option auth.addr.brick.allow *
option auth.addr.brick-ns.allow *
option transport.socket.nodelay on
option auth.ip.locks.allow *
subvolumes brick
end-volume
* file to test locking...
r...@gluster02:/mnt/glusterfs# cat locktest.pl
#!/usr/bin/perl
use Fcntl qw(:flock);
my $lock_file = 'lockfile';
open(LOCKFILE,">>$lock_file") or die "Cannot open $lock_file: $!\n";
print "Opened file $lock_file\n";
flock(LOCKFILE, LOCK_SH) or die "Can't get shared lock on $lock_file:
$!\n";
print "Got shared lock on file $lock_file\n";
sleep 2;
close LOCKFILE;
print "Closed file $lock_file\n";
exit;
*Test run from gluster02 using normal NFS mount:
r...@gluster02:/# mount <ip removed for security>:/mnt/test /mnt/test
r...@gluster02:/# cd /mnt/test
r...@gluster02:/mnt/test# ./locktest.pl
Opened file lockfile
Got shared lock on file lockfile
Closed file lockfile
*Test run from gluster02 using gluster exported NFS mount:
r...@gluster02:/# mount 74.81.128.17:/mnt/glusterfs /mnt/test
r...@gluster02:/# cd /mnt/test
r...@gluster02:/mnt/test# ./locktest.pl
Opened file lockfile
Can't get shared lock on lockfile:
No locks available
--
Ian Steinmetz
Comcast Engineering - Houston
713-375-7866
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
