Alright, who's the smart alleck that fixed NFS this last week? :) , WAS: Re: solid NFS patch #6 avail for -current - need testers files)
On Tue, 20 Apr 1999, Matthew Dillon wrote: NFS patch #6 is now available for -current. This patch has been extensively tested with NFS and with FFS+softupdates and has not screwed up yet, so I'm reasonably confident that it will not scrap whole filesystems :-) http://www.backplane.com/FreeBSD4/ Please remember to back-out all prior patches before applying this one. Note that the memory-zeroing code ( which is committed to -current ), is *correct* and should not be disabled. This patch is for CURRENT ONLY. Do not apply to -3.x unless you like seeing computer equipment melt! The only difference between patch #5 and patch #6 is that the VMIO directory backing mods have been removed. These mods worked, but appear to have resulted in an occassional softupdates panic during 'installworld'. It is more important for us to have a rock solid implementation then majorly optimized implementation, so... The patch will probably return later on when we figure out why it is causing softupdates to panic. Please post bug reports to -current or -hackers. .(14:57:37)(r...@thumper.reserved) /usr # mount /dev/da0s1a on / (local, soft-updates, writes: sync 598 async 27021) /dev/da0s1g on /home (local, soft-updates, writes: sync 25 async 679) /dev/da0s1f on /usr (local, soft-updates, writes: sync 738 async 42763) /dev/da0s1e on /var (local, soft-updates, writes: sync 106 async 1783) procfs on /proc (local) server:/usr/src on /usr/src server:/usr/obj on /usr/obj server:/home/ncvs on /home/ncvs /usr/src # for i in 1 2 3 4 5 6 ; do make world -j64 ; echo $i done.. ; done ../build.log nfs server server:/usr/obj: not responding nfs server server:/usr/obj: is alive again .(15:00:59)(r...@thumper.reserved) /usr # uptime 3:01PM up 6:55, 1 user, load averages: 1.72, 7.03, 7.34 .(15:01:47)(r...@thumper.reserved) /usr # grep ^[0-9] done build.log 1 done.. 2 done.. 3 done.. .(15:05:05)(r...@thumper.reserved) /usr # .(03:33:51)(r...@bright.reserved) /usr/obj # uptime 10:02AM up 14:40, 1 user, load averages: 0.18, 0.16, 0.14 yeah the clocks are not setup properly :) but otherwise i'm just gonna say HOLY SH*T you fixed NFS! :) Actually I just realized I'm not running nfsiod on the client, I just started up 8 of them (it's an SMP box). So far no problems. I'm using the default mount operations, as far as NFS server not responding messages, i have no clue, but the server is still up and i've seen that message happen when a lot of pressure is being put on an NFS server even though everything is fine. Normally i'd have client corruption and a rebooted server with both of them locked up sending out garbled RPC about now, but everything seems fine... great work! 2 questions I had: 1) you said you disabled partial writes that were causing these mmap() problems, they were causing problems because NFS had to muck with the structures directly in order to do zero copy? so if our NFS impelementation didn't do that it wouldn't be an issue probably. I know it's a good thing for speed and definetly essensial, but i'm not sure i understand NFS and the FS getting out of sync. 2) at BAFUG 2 or 3 months ago I, *cough* attempted to keep up with you an Julian talking about VM issues. :) Something you guys brought up was problems with mmap() + read()/write() no staying in sync and requireing an msync() to correctly syncronize. I really didn't understand how this could happen except recently I figured that my first question could be the answer. Does this problem only happen on NFS mounted dirs? Is it fixed? thanks again, -Alfred To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-current in the body of the message
Re: Alright, who's the smart alleck that fixed NFS this last week? :) , WAS: Re: solid NFS patch #6 avail for -current - need testers files)
:2 questions I had: : :1) you said you disabled partial writes that were causing these :mmap() problems, they were causing problems because NFS had to :muck with the structures directly in order to do zero copy? : so if our NFS impelementation didn't do that it wouldn't be :an issue probably. I know it's a good thing for speed and definetly :essensial, but i'm not sure i understand NFS and the FS getting out :of sync. The problem w/ the partial writes has to do with cache coherency between the server, the client's VFS subsystem ( read() and write() ), and the client's VM subsystem ( mmap() ). NFS implemented the notion of unaligned valid and dirty range using struct buf's b_validoff, b_validend, b_dirtyoff, and b_dirtyend fields in order to keep track of partial writes without having to read-in the rest of the buffer. The implementation was very fragile and failed to address a number of combination situations that would occur with mmap(), read(), and write(). This in turn lead to a series of problems and, further, lead to the situation where we would fix unrelated bugs in the VM system and cause NFS to break. I finally gave up on it. What NFS does now is optimize only two write situations: (1) when a write covers the entire buffer, e.g. an 8K+ write on an 8K boundry. And (2) piecemeal writes in the write-append case. Both cases allow us to mark the buffer as essentially being fully valid without having to mess with valid and dirty ranges. We use buf-b_bcount to handle the file EOF case and resize it rather then try to use b_validoff/b_validend. Thus, b_validoff/b_validend have been completely removed. b_dirtyoff/b_dirtyend have been left intact in order to allow us to support piecemeal write RPC's. This is different from the piecemeal write optimizations we were doing before. In this case, we are able to support piecemeal writes in the middle of the file that go into *PRELOADED* buffers. That is, A read-merge-write case. The original code attempted to do piecemeal writes without the read-before resulting in the partially invalid, partially dirty buffer. Now we only allow piecemeal writes to occur in fully-valid buffers. While we could theoretically discard the dirty range and simply writeback the entire buffer when a modification is made to part of it, we keep the dirty range in order to *only* write the portion of the buffer that the explicit write() covered. This is done for cache coherency reasons. For example, take the situation where two different client machines do a seek/write to different portions of the same server-backed NFS file, where the two areas abut each other. Say one client writes 2 bytes at seek offset 10 and the second client writes 2 bytes at seek offset 12. As long as the areas are not overlapping, we want this type of operation to work properly and not scramble the data on the server even if the client's idea of the state of the date is not coherent. :2) at BAFUG 2 or 3 months ago I, *cough* attempted to keep up with you :an Julian talking about VM issues. :) Something you guys brought up :was problems with mmap() + read()/write() no staying in sync and requireing :an msync() to correctly syncronize. I really didn't understand how this :could happen except recently I figured that my first question could be :the answer. Does this problem only happen on NFS mounted dirs? Is it :fixed? : :thanks again, :-Alfred This should not be an issue any more for either UFS or NFS. If people find that it is an issue, there's a bug somewhere that needs to be addressed. This *was* an issue for NFS prior to the patch set. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-current in the body of the message
Re: Alright, who's the smart alleck that fixed NFS this last week? , :) , WAS: Re: solid NFS patch #6 avail for -current - need testers , files)
On Wed, 21 Apr 1999, Matthew Dillon wrote: :2 questions I had: : :2) at BAFUG 2 or 3 months ago I, *cough* attempted to keep up with you :an Julian talking about VM issues. :) Something you guys brought up :was problems with mmap() + read()/write() no staying in sync and requireing :an msync() to correctly syncronize. I really didn't understand how this :could happen except recently I figured that my first question could be :the answer. Does this problem only happen on NFS mounted dirs? Is it :fixed? : :thanks again, :-Alfred This should not be an issue any more for either UFS or NFS. If people find that it is an issue, there's a bug somewhere that needs to be addressed. This *was* an issue for NFS prior to the patch set. Ok, so is this what you and Julian were discussing about coherency issues? And it is fixed... cool. :) -Alfred To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-current in the body of the message
