Yes, that's what'll happen if you use sync. One way around using sync and getting decent write performance is to use rsize and wsize >= 128kb, but then you'll lose small file performance since you're moving larger blocks over the wire. The problem I've seen with all the EMC solutions I've tested is that NFSv3 client (read) data cache usage is broken unless you use sync or you set the EMC server to force NFS unstable writes to filesync or datasync rather than the default of an unstable NFS write (async).
What I mean by broken client (read) cache usage is that if you write a file out to the server which fits in memory on the client (since the client buffers the write to local memory before sending it to the server), then if you try to read the file back and it's still in local client memory, the EMC server will do something with the client that causes it to read the entire file back again over the wire, rather than just getting it from local memory (really fast). Between Linux NFS servers and clients or other high end NAS solutions I've tested the client will read the file out of cache if it exists and if it's unchanged, which is the correct behavior. dd is actually a pretty good way of determining whether several NFS stacks are "broken" in this regard, I've written a nice wrapper around it https://code.google.com/p/nfsspeedtest/ On Mon, Aug 27, 2012 at 9:19 AM, Gerhardus Geldenhuis <gerhardus.geldenh...@gmail.com> wrote: > Hi > I am debugging NFS performance problems at the moment and going through a > steep learning curve with regards to all its inner workings. > > In short we are mounting EMC provided NFS storage and seeing a massive > difference when mounting with and without the sync option. > > Without the sync option I can achieve 90MB/s without any tuning, with sync > option turned on that falls to 4MB/s. That seems to slow... any thoughts and > experience on this would be appreciated. > These values were obtained by doing a dd if=/dev/zero of=<file onnfs share> > bs=1M count=1024. I am aware that I need to optimize for specific use case > with regards to concurrency, file size, writes, rewrites, reads, rereads > etc. However the performance penalty for sync seems excessive and I wanted > to know if that is a shared experience. > > My understanding of sync is basically that in the case of NFS, it will wait > for a confirmation from the NFS server that the data has been written to > disk before acknowledging the write locally, so being "very certain". My > understanding could be flawed and I would appreciate anyone correcting me on > that and/or pointing me to more reading which I would happily do. > > I am a bit confused however about what the man pages are saying: > man mount says: > the sync option today has effect only for ext2, ext3, fat, vfat and ufs > but when I change it I can see a big difference in performance so the man > page is out of date... or is there something I am misreading? > > man 5 nfs, does not make mention of sync. > > OS: RHEL 5.8 > Kernel: 2.6.18-308.4.1.0.1.el5 > NFS: > nfs-utils-lib-1.0.8-7.9.el5 > nfs-utils-1.0.9-60.0.1.el5 > mount parameters: > nfs4 > rw,bg,hard,nointr,rsize={16384,32768,65536},wsize={16384,32768,65536},suid,timeo=600,_netdev > 0 0 > > I am currently running iozone and changing various mount parameters to see > how they effect performance. iozone is arguably a better test of performance > than dd > > Regards > > -- > Gerhardus Geldenhuis > > _______________________________________________ > rhelv5-list mailing list > rhelv5-list@redhat.com > https://www.redhat.com/mailman/listinfo/rhelv5-list _______________________________________________ rhelv5-list mailing list rhelv5-list@redhat.com https://www.redhat.com/mailman/listinfo/rhelv5-list
