For me at least on gfs, I'm using lock_dlm and I was getting like 1.3-1.5 MB/s transfer rates on samba. I know you said you using lock_nolock but I say give ping_pong a shot:
http://wiki.samba.org/index.php/Ping_pong Another thing to try to see if locking is the culprit is in smb.conf set posix locking = no and try a transfer. That's why my transfer rates were so bad. With ping_pong I noticed 94-100 locks per seconds. Turns out gfs_controld has 100 locks per/s as the default and can be lifted by -l0 flag. With that on, I then got like 25-30k locks per/s and my transfer rates went up with samba to like 8 - 9 MB/s (on 100mbit lan). I also tried the plock optimization that I believe was introduced not to many versions ago and the rate when up to like 100k and transfer rate got a little better. It's probably best to put some lower limit as a side effect is more network load and lock coordination amongst the nodes, I'm trying to find a good balance with my setup. I tried all this in a test environment btw and I'm still tweaking/learning about it :). I mean that was my personal experience with gfs (not gfs2, don't know if it's the same). Hope this helps. Regards, Arwin -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Andrew A. Neuschwander Sent: Thursday, April 23, 2009 11:03 AM To: linux clustering Subject: [Linux-cluster] nfs/smb slow on gfs2 I have a local only (lock_nolock) gfs2 filesystem which is served to clients via smb and nfs v3. Crawling around the filesystem is extremely slow over smb and nfs. Stat, ls and find over the network take many orders of magnitude longer than I'd expect. The same tasks are very fast locally on the gfs2 filesystem. Also, for comparison, I created an ext3 on the same server and exported it via smb and nfs. The same task over nfs or samba on ext3 is nearly as fast as doing the task locally. Is there any way to optimized the interaction between smb/nfs and gfs2? I'm using gfs2 because I needed a filesystem over 8TB. I'm on CentOS5.3 (2.6.18-128.1.6.el5). The storage is locally attached. It is two hardware RAID6 (11 disks each + 1 hotspare) arrays striped together with software raid, then managed with lvm2. Both the ext3 and gfs2 are on this storage. Large file sequential reads are fast. For example, dd if=file-on-nfs of=/dev/null will where file-on-nfs is an 80GB file stored on the gfs2 will max out a GigE connection. Any insight or advice on speeding up these operations would be appreciated. Thanks, -Andrew -- Andrew A. Neuschwander, RHCE Systems/Software Engineer College of Forestry and Conservation The University of Montana http://www.ntsg.umt.edu [email protected] - 406.243.6310 -- Linux-cluster mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-cluster -- Linux-cluster mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-cluster
