Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
In article <[EMAIL PROTECTED]>, Sergey Babkin <[EMAIL PROTECTED]> writes > >By the way the journaling filesystems don't neccessary guarantee that >you won't need fsck: for example, if VXFS crashes at a particularly >bad moment, it will require you to do "fsck -o full" which is as slow >as the fsck on traditional UFS. JFS still scores against traditional Unix file systems on large volumes, (e.g. Terabytes), as it requires very small amounts of virtual memory during a full fsck. ttfn, Tony To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
On Tue, 18 Dec 2001, Brandon D. Valentine wrote: > On Tue, 18 Dec 2001, Mike Bristow wrote: > > >I suspect that the background fsck[1] that's available in FreeBSD-current > >fits the bill just as well as JFS or XFS - and I'll also bet that it'll > >be available in a FreeBSD-release before I'd trust data to a port of > >JFS or XFS. > > This is a killer feature. Has anyone decided whether snapshots and > background fsck will ever be backported to the RELENG_4 branch or are > they destined for 5.0? In a word (or two): highly unlikely. This code has been considered experimental for a while now, and I expect that it will remain so. While it has been gradually improving stability (it no longer toasts your system when you send a kill signal to fsck_ffs in the background), a number of usability factors are still being addressed. Kirk recently committed several performance improvements that (apparently) result in a far more usable system during the background fsck. Previously, my system was available, but largely unuseful, during the background fsck. This code relies on the FFS snapshot feature, which is also not as widely tested, and has some compatibility concerns. If the support for snapshots hasn't yet been MFC'd to -STABLE fsck, we may want to consider doing so; last time I checked, if a snapshot was found by RELENG_4's fsck, it would be rather sadly removed with some unhappiness from fsck. As such, I'd probably resist efforts to MFC this code, and just go for inclusion in 5.0-RELEASE. We'll need to give it a lot of testing however. :-) Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
Alfred Perlstein wrote: > > By the way the journaling filesystems don't neccessary guarantee that > > you won't need fsck: for example, if VXFS crashes at a particularly > > bad moment, it will require you to do "fsck -o full" which is as slow > > as the fsck on traditional UFS. > > Yeah, but that's not mentioned in the whitepaper! :) Your insane humor quotient is very high today... Actually, this is mentioned in the white papers of all journalling FSs, but is generally glossed over with application specific hardware that is missing on PCs, which will record the cause of the failure across a reboot, and will throw a chock in front of the wheels before a bad write on a power failure... something IDE drives fail to do, but SCSI drives do not (or did not, until recently). Of course, you can't just use PC CMOS for this because of the lack of DC hold up time and AC fail notification in standard PC power supplies. You owe the Oracle your first born child, and , because of the GPL, anyone who marries your first born child owes the Oracle their first born child, and so on, recursively and eternally, forever after. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
* Sergey Babkin <[EMAIL PROTECTED]> [011218 19:45] wrote: > Dan Nelson wrote: > > > > In the last episode (Dec 18), Mike Bristow said: > > > I suspect that the background fsck[1] that's available in FreeBSD-current > > > fits the bill just as well as JFS or XFS - and I'll also bet that it'll > > > be available in a FreeBSD-release before I'd trust data to a port of > > > JFS or XFS. > > > > The problems with a background fsck is you still have to run fsck, > > which can take 10 minutes on a large volume when it's idle, and who > > By the way the journaling filesystems don't neccessary guarantee that > you won't need fsck: for example, if VXFS crashes at a particularly > bad moment, it will require you to do "fsck -o full" which is as slow > as the fsck on traditional UFS. Yeah, but that's not mentioned in the whitepaper! :) -- -Alfred Perlstein [[EMAIL PROTECTED]] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' http://www.morons.org/rants/gpl-harmful.php3 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
Dan Nelson wrote: > > In the last episode (Dec 18), Mike Bristow said: > > I suspect that the background fsck[1] that's available in FreeBSD-current > > fits the bill just as well as JFS or XFS - and I'll also bet that it'll > > be available in a FreeBSD-release before I'd trust data to a port of > > JFS or XFS. > > The problems with a background fsck is you still have to run fsck, > which can take 10 minutes on a large volume when it's idle, and who By the way the journaling filesystems don't neccessary guarantee that you won't need fsck: for example, if VXFS crashes at a particularly bad moment, it will require you to do "fsck -o full" which is as slow as the fsck on traditional UFS. -SB To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
In the last episode (Dec 18), Mike Bristow said: > I suspect that the background fsck[1] that's available in FreeBSD-current > fits the bill just as well as JFS or XFS - and I'll also bet that it'll > be available in a FreeBSD-release before I'd trust data to a port of > JFS or XFS. The problems with a background fsck is you still have to run fsck, which can take 10 minutes on a large volume when it's idle, and who knows how long as a background process when the system's up. It might not even finish at all if a user starts modifying a large file, causing the snapshot file that the background fsck is using to grow and fill up the filesystem. Unlikely, but possible if your disk is almost full already. -- Dan Nelson [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
On Thu, Dec 13, 2001 at 04:39:58AM -0500, Brandon D. Valentine wrote: > On Wed, 12 Dec 2001, Matthew Dillon wrote: > > >All I can say is... holy shit! > > Dude, you kick ass. At work I've been dealing with Linux's crappy NFS > implementation for years, while FreeBSD has always been pretty damn good > by comparison. Linux finally got a decent amount of performance under > 2.4 (which finally does NFSv3 to hosts other than other Linux boxen), > but it still can't touch the FreeBSD NFS implementation. The more > robust you make it the easier it is for me to argue for deployment of > more FreeBSD systems in NFS server roles. The only advantage Linux has > got right now is XFS, which is admittedly a pretty large advantage on > multi terabyte filesystems where fsck is impossible. I'm guessing that the real requirment here is is "when the system is turned on after an unclean shutdown (eg, power failure), it should be able to export it's NFS filesystems quickly". I suspect that the background fsck[1] that's available in FreeBSD-current fits the bill just as well as JFS or XFS - and I'll also bet that it'll be available in a FreeBSD-release before I'd trust data to a port of JFS or XFS. [1] If you've missed it, the basic idea is: for $fs in $all_filesystems ; do if is_a_softupdate_filesystem($fs) ; then fsck $fs & else fsck $fs fi done except it happens in fsck itself, rather than a shell script. -- Mike Bristow, embonpointful, but not managerial, damnit. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
I'm trying to get the license issue clarified, then it can go in /usr/src/tools/regression. - Jordan > Jordan Hubbard <[EMAIL PROTECTED]> writes: > > > Guy Harris of NetApp sent me a whole mess-o-changes to it and when I > > went to forward them to you, I found that I must have been in > > delete-o-matic mode at some point earlier in my inbox since it was > > gone. I've requested that he send them to me again and will forward > > them to you once I get a copy again. Whoops! > > Would it be worth making a port for this tool? It sounds like it's > too important to get lost in a mailing list archive. There's a > precedence set by having /usr/ports/sysutils/crashme. :-) > > -Dom > > -- > | Semantico: creators of major online resources | > | URL: http://www.semantico.com/ | > | Tel: +44 (1273) 72 | > | Address: 33 Bond St., Brighton, Sussex, BN1 1RD, UK. | To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
In article <[EMAIL PROTECTED]>, Brandon D. Valentine <[EMAIL PROTECTED]> writes > [snip] >but it still can't touch the FreeBSD NFS implementation. The more >robust you make it the easier it is for me to argue for deployment of >more FreeBSD systems in NFS server roles. The only advantage Linux has >got right now is XFS, which is admittedly a pretty large advantage on >multi terabyte filesystems where fsck is impossible. That is what I wanted to hear, an unambiguous argument that a solid implementation of JFS would be useful to some user segment. Tony To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Apple and BSD (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
Aloha! Sorry to jump in here but.. I've read the thread starting with Jordans posting of the Apple testcase code with great interest. To me this has been one of the most interesting and reassuring readings in hackers for a long time. (Yes, I'm a lurker here). First, I take the testcase code as a sign that Apple does indeed give back to the Open Source community. I've read some concerns that Apple as a long time proprietary developer might not do that, but this is a good sign indeed. Second, the speed at which the FreeBSD community, especially Matt used the testcase to iron out, hunt down bugs and respond to the problems the testcase highlighted have been truly astonishing. It really shows how fast and well a community can react and be flexible and efficient. As a simple user and sysadm I'm impressed and grateful. Couln't this whole thing be documented and highlighted in the monthly development newsletter or something? It's way to good not to be written down. Just my 1 Euro. -- Med vänlig hälsning, Cheers! Joachim Strömbergson Joachim Strömbergson - ASIC designer, nice to *cute* animals. snail: phone: mail & web: Sävenäsgatan 5A+46 31 - 27 98 47 [EMAIL PROTECTED] 416 72 Göteborg+46 733 75 97 02 www.ludd.luth.se/~watchman To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
JFWIW, you can build fsx with minimal or no changes on Windows with David Korn's UWIN kit. All of the other posix-y kits have internal problems that will cause spurious failures. If you want to use Windows boxes as test clients (probably a good idea) this is fairly important... > > I gave out fsx source code at the recent CIFS (SMB) plugfest. If I make > > the 2002 Connectathon I'll give it out there too. I don't test it on > > Windows so those defines may be in need of repair. Please send me any > > patches or cool additions. > > Guy Harris of NetApp sent me a whole mess-o-changes to it and when I > went to forward them to you, I found that I must have been in > delete-o-matic mode at some point earlier in my inbox since it was > gone. I've requested that he send them to me again and will forward > them to you once I get a copy again. Whoops! > > - Jordan -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
> I gave out fsx source code at the recent CIFS (SMB) plugfest. If I make > the 2002 Connectathon I'll give it out there too. I don't test it on > Windows so those defines may be in need of repair. Please send me any > patches or cool additions. Guy Harris of NetApp sent me a whole mess-o-changes to it and when I went to forward them to you, I found that I must have been in delete-o-matic mode at some point earlier in my inbox since it was gone. I've requested that he send them to me again and will forward them to you once I get a copy again. Whoops! - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
At 8:19 PM -0800 12/12/01, Jordan Hubbard wrote: >> To be clear, what exactly are you doing? >> >> It sounds like you're exporting something from freebsd, mounting it on OSX >> and running this tool on OSX against the filesystem exported from freebsd ? >> >> If so, What mount options? NFSv2 or v3? > >That is correct. As to the NFS options used, I honestly couldn't say >since I'm getting at the filesystem through Netinfo and that's handled >by OS X's automount daemon, that having no relation whatsoever to AMD >and hence no amd.conf file or anything else I can easily look at to >determine how it's being mounted. Maybe Mike knows more about how to >find this out - he's not in management. :) In the absence of mount-options the OS X automount tries v3 first, followed by v2 if v3 appears not supported by the server. -- Conrad Minshall, [EMAIL PROTECTED], 408 974-2749 Apple Computer, Mac OS X Core Operating Systems To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
At 2:56 PM -0800 12/12/01, Jordan Hubbard wrote: >> The only thing I get is a math exception because "closeprob" is zero >> since no -c option was given. >> >> Can you provide some sample parameters please ? > >Hmmm, how strange, now that I look at the code it's obvious that a >divide by zero will occur with a zero closeprob and the docs state the >default to be "infinity", which is obviously not the case. The >strange part is that I ran this on freebsd.apple.com, which is running >4.4-stable, with one parameter (the filename) exactly as I pasted in >the usage instructions before. Perhaps all this time spent living >next to the Macintosh in my office has induced that copy of FreeBSD to >be more "friendly" and mask simple math errors. :-) > >In any case, -c 1 appears to work just fine. That gives a close/open between each "operation". Better is to fix the fsx.c source by inserting an "if" in main()... diff -u -d -b -w -r1.21 fsx.c --- fsx.c 2001/12/11 23:27:20 1.21 +++ fsx.c 2001/12/15 15:31:19 @@ -701,6 +701,7 @@ testcalls++; + if (closeprob) closeopen = (rv >> 3) < (1 << 28) / closeprob; if (debugstart > 0 && testcalls >= debugstart) -- Conrad Minshall, [EMAIL PROTECTED], 408 974-2749 Apple Computer, Mac OS X Core Operating Systems To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
On Thu, Dec 13, 2001 at 01:40:46PM -0800, Matthew Dillon wrote: > > :Matt, > : > :what the hell, this seems to very near by a problem I wanted to > :report since a week: > : > :in a data acquisition I have a write process writing to a file > :backed shared mmapped ringbuffer. There can be several reader > :processes on this this ringbuffer. Now once i killed the writer for > :resizing of the ringbuffer and forgot about the readers. The writer > :truncated the database without unlinking it before. This lead the > :readers to be running for ever, it seemed so at least. After > :attaching with gdb I saw, that they were only page faulting nothing > :more, for ever > : > :Something similar I saw with netscape going mad. > : > :cheers, Thomas > > That's something else. There's no OS bug there. When you mmap() > a file only those pages that are within the file's boundries are > valid. So if you ftruncate() the file then all the pages occuring > after the (new) file EOF will become invalid and BUSfault if the > process touches them. > > You touched upon the correct solution... remove() the file instead > of ftruncate()ing it. The file's data then remains intact for the > processes still referencing it. > > The readers must be catching SIGBUS and retrying (not exiting), > causing them to run in a signal loop forever. This is a case of > bad programming. I've seen it before... there was a popular IRC > bot back in my BEST days which constantly got itself into infinite > loops because the guy who wrote it installed a signal handler for > SIGBUS. > > -Matt > Matthew Dillon > <[EMAIL PROTECTED]> well, I know, that this was a bug in my software, not to unlink the file first and then truncating :-). But SIGBUS was not catched in the readers. Will try to reproduce it. Thomas To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
:Matt, : :what the hell, this seems to very near by a problem I wanted to :report since a week: : :in a data acquisition I have a write process writing to a file :backed shared mmapped ringbuffer. There can be several reader :processes on this this ringbuffer. Now once i killed the writer for :resizing of the ringbuffer and forgot about the readers. The writer :truncated the database without unlinking it before. This lead the :readers to be running for ever, it seemed so at least. After :attaching with gdb I saw, that they were only page faulting nothing :more, for ever : :Something similar I saw with netscape going mad. : :cheers, Thomas That's something else. There's no OS bug there. When you mmap() a file only those pages that are within the file's boundries are valid. So if you ftruncate() the file then all the pages occuring after the (new) file EOF will become invalid and BUSfault if the process touches them. You touched upon the correct solution... remove() the file instead of ftruncate()ing it. The file's data then remains intact for the processes still referencing it. The readers must be catching SIGBUS and retrying (not exiting), causing them to run in a signal loop forever. This is a case of bad programming. I've seen it before... there was a popular IRC bot back in my BEST days which constantly got itself into infinite loops because the guy who wrote it installed a signal handler for SIGBUS. -Matt Matthew Dillon <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
: :I can't seem to get a crash dump out of it, but the panic I'm seeing is: : :panic: allocdirect_check: old 30246736 ! = new 30246736 || ibn 4 >= 12 : :For some reason the dump also dies, resettting the ata0 bus over and over :but not writing out to swap. : :- Jordan That's the softupdates panic I fixed in the patch set. There are actually three conditions to the panic and the one that isn't being listed is the one that is causing the panic. Weird that you couldn't dump, but don't worry about it, it's been dealt with. -Matt Matthew Dillon <[EMAIL PROTECTED]> :> :> What panic are you getting? :> :> -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
On Thu, 13 Dec 2001, Jordan Hubbard wrote: > I can't seem to get a crash dump out of it, but the panic I'm seeing is: > > panic: allocdirect_check: old 30246736 ! = new 30246736 || ibn 4 >= 12 > > For some reason the dump also dies, resettting the ata0 bus over and over > but not writing out to swap. > > - Jordan Was it allocdirect_merge perhaps? It looks like the debugging info it gives there is incomplete, three things can actually cause the panic to trigger. This panic is in the soft updates code; have you tried running the test with the soft updates patch applied? Mike "Silby" Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
On Thu, Dec 13, 2001 at 02:58:28AM -0800, Matthew Dillon wrote: > > @#$@#$ crap. I think I found a dirty-mmap edge case with truncation. > It requires a change to vm_page_set_validclean(), which of course is > one of the core routines in the VM system. > > Basically what happens is that ftruncate() calls vnode_pager_setsize() > which eventually calls vm_page_set_validclean(). > > If you happened to mmap() the truncation point shared R+W and > dirty it, then truncate to something that isn't a multiple DEV_BSIZE.. > for example, if you were to truncate to an offset of '10', and a buffer Matt, what the hell, this seems to very near by a problem I wanted to report since a week: in a data acquisition I have a write process writing to a file backed shared mmapped ringbuffer. There can be several reader processes on this this ringbuffer. Now once i killed the writer for resizing of the ringbuffer and forgot about the readers. The writer truncated the database without unlinking it before. This lead the readers to be running for ever, it seemed so at least. After attaching with gdb I saw, that they were only page faulting nothing more, for ever Something similar I saw with netscape going mad. cheers, Thomas To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
I can't seem to get a crash dump out of it, but the panic I'm seeing is: panic: allocdirect_check: old 30246736 ! = new 30246736 || ibn 4 >= 12 For some reason the dump also dies, resettting the ata0 bus over and over but not writing out to swap. - Jordan > : > :> I should point out that FSX can be used against any filesystem, and > :> that there are reports locally (at Apple) that it's great for killing > :> FreeBSD machines. I wasn't able to reproduce this when I tried, but I > :> may not have let it run long enough. > : > :Oh, it blows freebsd.apple.com right out of the water with a kernel > :panic after running for just 3 seconds from an OS X box on the same > :LAN segment. :) > : > :- Jordan > > What panic are you getting? > > -Matt > Matthew Dillon > <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
:> And if you hadn't heard, Matt just fixed a couple of bugs in the tcp :> stack which improves NFS greatly. It sounds like after this round of :> NFS fixes, the first answer to NFS questions should be: Upgrade to :> 4.5! : :I don't even bother with TCP mounts; my default amd rule says :proto=udp. Is there any reason to add the overhead of the TCP stack if :you're not leaving your own ethernet? : :You should be able to easily saturate a 100mbit link with FreeBSD 4.* :machines, and I can do 15-20MB/sec with Netgear GA620 gigabit nics (SMP :2 x pIII/600). : :-- : Dan Nelson : [EMAIL PROTECTED] I think there's a trade-off. TCP mounts deal with heavy parallel client loads better then UDP because they do real congestion and streaming control whereas NFS's UDP implementation fakes it. A UDP mount will work better for a large parallel load from a single client. I tend to use both types of mounts but I personally prefer TCP mounts over UDP because they are more secure and easier to get through a firewall. -Matt Matthew Dillon <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
* void <[EMAIL PROTECTED]> [011213 11:00] wrote: > Has anyone brought this code to the attention of the NetBSD people? > I imagine that they would be interested. If not, I will forward it > along myself, as soon as I determine the appropriate list. (List > recommendations from the dual citizens here are welcome.) tech-kern might be appropriate. -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
: :Matt didnt feel the fixes were primarily performance related, just bug :fixes. I meant just the last two days worth of NFS stuff are just bug fixes. The TCP fixes from earlier in the month are definitely performance fixes. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
On Thu, 13 Dec 2001, Geoff Mohler wrote: > Matt didnt feel the fixes were primarily performance related, just bug > fixes. Right, the fact that they are bug fixes is what seems to be the most powerful reason to recommend upgrading. Mike "Silby" Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
On Thu, 13 Dec 2001, Dan Nelson wrote: > In the last episode (Dec 13), Mike Silbersack said: > > And if you hadn't heard, Matt just fixed a couple of bugs in the tcp > > stack which improves NFS greatly. It sounds like after this round of > > NFS fixes, the first answer to NFS questions should be: Upgrade to > > 4.5! > > I don't even bother with TCP mounts; my default amd rule says > proto=udp. Is there any reason to add the overhead of the TCP stack if > you're not leaving your own ethernet? > > You should be able to easily saturate a 100mbit link with FreeBSD 4.* > machines, and I can do 15-20MB/sec with Netgear GA620 gigabit nics (SMP > 2 x pIII/600). > > -- > Dan Nelson > [EMAIL PROTECTED] I'm not a NFS user, so I can't comment on how NFS is normally used... but my guess would be that while UDP may be faster when you're benchmarking two computers, it'll cause a lot more congestion when you have 200 computers. Mike "Silby" Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
For some people, the overhead is an acceptible tradeoff to redundancy. Ever since Cisco released thier 6500 10/100 blades that to crappy buffering between a Gigabit NFS server (could be anything, just an example) and an 100Mbit client, people have somewhat been adding that overhead to thier CPU and data-rate budgets as "acceptable losses". On Thu, 13 Dec 2001, Dan Nelson wrote: > In the last episode (Dec 13), Mike Silbersack said: > > On Wed, 12 Dec 2001, Matthew Dillon wrote: > > > Geoff Mohler wrote: > > > :Are there any hidden secrets to eeking out more performance from > > > :the BSD NFS client (other than version types and the normal fstab > > > :tweaks). > > > > And if you hadn't heard, Matt just fixed a couple of bugs in the tcp > > stack which improves NFS greatly. It sounds like after this round of > > NFS fixes, the first answer to NFS questions should be: Upgrade to > > 4.5! > > I don't even bother with TCP mounts; my default amd rule says > proto=udp. Is there any reason to add the overhead of the TCP stack if > you're not leaving your own ethernet? > > You should be able to easily saturate a 100mbit link with FreeBSD 4.* > machines, and I can do 15-20MB/sec with Netgear GA620 gigabit nics (SMP > 2 x pIII/600). > > -- > Dan Nelson > [EMAIL PROTECTED] > --- Geoff Mohler To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
Has anyone brought this code to the attention of the NetBSD people?
I imagine that they would be interested. If not, I will forward it
along myself, as soon as I determine the appropriate list. (List
recommendations from the dual citizens here are welcome.)
On Wed, Dec 12, 2001 at 08:23:56AM -0800, Jordan Hubbard wrote:
> It came up in a meeting today at Apple just how fragile the BSD NFS
> implementation was before significant work was put in to stabilizing it,
> and in that discussion came up a little test tool written originally by
> Avie Tevanian and subsequently improved by one of the folks here.
>
> This tool basically tries to do everything it can (legally) to confuse an
> NFS server. It seeks around, does I/O to and truncates/changes the size
> of a test file, all while doing everything it can to detect data corruption
> or other signs of misbehavior which might result from out-of-order replies
> or any other previously-observed NFS pathology. Very few NFS implementations
> apparently survive this test and FreeBSD's is no exception. The sources are
> provided below, courtesy of Avie, for the education and enjoyment(?) of
> anyone who's motivated to play with (or even pretends to understand) NFS.
>
> Usage:
> cc fsx.c -o fsx
> ./fsx /some/nfs/mounted/scratchfile
> [ ** kaboom! ** ]
>
> I'm also trying to determine which of the fixes Apple has made to NFS might
> be adapted to FreeBSD, something which is made more difficult by the fact
> that much of the code was taken straight from 4.4 Lite some time back and
> both operating systems have diverged significantly since then. Anyone
> really keen on investigating this further themselves on it can also go to
> http://www.opensource.apple.com/projects/darwin and register themselves
> online (it's easy) to access the Darwin CVS repository, the module in
> question being "xnu" (the Darwin kernel). Thanks.
>
> - Jordan
>
> /*
> *Copyright (C) 1991, NeXT Computer, Inc. All Rights Reserverd.
> *
> *File: fsx.c
> *Author: Avadis Tevanian, Jr.
> *
> *File system exerciser.
> *
> *Rewritten 8/98 by Conrad Minshall.
> */
>
> #include
> #include
> #ifdef _UWIN
> # include
> # include
> # include
> # include
> # define MAP_FILE 0
> #else
> # include
> #endif
> #include
> #include
> #include
> #include
> #include
> #include
> #include
> #include
> #include
> #include
> #include
>
> #define NUMPRINTCOLUMNS 32/* # columns of data to print on each line */
>
> /*
> *A log entry is an operation and a bunch of arguments.
> */
>
> struct log_entry {
> int operation;
> int args[3];
> };
>
> #define LOGSIZE 1000
>
> struct log_entry oplog[LOGSIZE]; /* the log */
> int logptr = 0; /* current position in log */
> int logcount = 0; /* total ops */
>
> /*
> *Define operations
> */
>
> #define OP_READ 1
> #define OP_WRITE 2
> #define OP_TRUNCATE 3
> #define OP_CLOSEOPEN 4
> #define OP_MAPREAD5
> #define OP_MAPWRITE 6
> #define OP_SKIPPED7
>
> #ifndef PAGE_SIZE
> #define PAGE_SIZE 4096
> #endif
> #define PAGE_MASK (PAGE_SIZE - 1)
>
> char *original_buf; /* a pointer to the original data */
> char *good_buf; /* a pointer to the correct data */
> char *temp_buf; /* a pointer to the current data */
> char *fname; /* name of our test file */
> int fd; /* fd for our test file */
>
> off_t file_size = 0;
> off_t biggest = 0;
> char state[256];
> unsigned long testcalls = 0; /* calls to function "test" */
>
> unsigned long simulatedopcount = 0; /* -b flag */
> int closeprob = 0; /* -c flag */
> int debug = 0; /* -d flag */
> unsigned long debugstart = 0; /* -D flag */
> unsigned long maxfilelen = 256 * 1024;/* -l flag */
> int sizechecks = 1; /* -n flag disables them */
> int maxoplen = 64 * 1024; /* -o flag */
> int quiet = 0; /* -q flag */
> unsigned long progressinterval = 0; /* -p flag */
> int readbdy = 1;/* -r flag */
> int style = 0; /* -s flag */
> int truncbdy = 1; /* -t flag */
> int writebdy = 1; /* -w flag */
> long monitorstart = -1; /* -m flag */
> long monitorend = -1;/* -m flag */
> int lite = 0; /* -L flag */
> long numops = -1;/* -N flag */
> int randomoplen = 1;/* -O flag disables it */
> int seed = 1; /* -S flag */
> int mapped_writes = 1; /* -W flag disables */
> int mapped_reads = 1; /* -R flag disables it */
> int fsxgoodfd = 0;
> FILE *fs
Re: NFS: How to make FreeBSD fall on its face in one easy step
Matt didnt feel the fixes were primarily performance related, just bug fixes. On Thu, 13 Dec 2001, Mike Silbersack wrote: > > On Wed, 12 Dec 2001, Matthew Dillon wrote: > > > :I suppose while were on the topic.. > > : > > :Are there any hidden secrets to eeking out more performance from the BSD > > :NFS client (other than version types and the normal fstab tweaks). > > : > > :Im the CS Labs manager at NetApp..and Im always trying to store away a > > :secret here or there when someone comes to me with a problem in the field. > > : > > :FreeBSD since v2..rock on! > > > > For extreme performance there are some zero-copy patches floating around > > which have not been integrated into the main tree. Generally, though, > > your NFS performance is going to be ultimately limited by your server's > > disk performance. > > > > -Matt > > Matthew Dillon > > And if you hadn't heard, Matt just fixed a couple of bugs in the tcp stack > which improves NFS greatly. It sounds like after this round of NFS fixes, > the first answer to NFS questions should be: Upgrade to 4.5! > > Mike "Silby" Silbersack > > --- Geoff Mohler To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
In the last episode (Dec 13), Mike Silbersack said: > On Wed, 12 Dec 2001, Matthew Dillon wrote: > > Geoff Mohler wrote: > > :Are there any hidden secrets to eeking out more performance from > > :the BSD NFS client (other than version types and the normal fstab > > :tweaks). > > And if you hadn't heard, Matt just fixed a couple of bugs in the tcp > stack which improves NFS greatly. It sounds like after this round of > NFS fixes, the first answer to NFS questions should be: Upgrade to > 4.5! I don't even bother with TCP mounts; my default amd rule says proto=udp. Is there any reason to add the overhead of the TCP stack if you're not leaving your own ethernet? You should be able to easily saturate a 100mbit link with FreeBSD 4.* machines, and I can do 15-20MB/sec with Netgear GA620 gigabit nics (SMP 2 x pIII/600). -- Dan Nelson [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
On Wed, 12 Dec 2001, Mike Smith wrote: > Many of the key issues in making OS X NFS work were related to its > interaction with the UBC and the subtly different VFS semantics, > although the same issues probably exist in different form in the > FreeBSD code. I get dragged into some really shocking corridor > discussions every now and then. 8) > > = Mike I just thought of something regarding fsx.c - I think it perhaps needs another set of test vectors added. There's a third way we read files now: through sendfile(). If someone has time, it might be worth adding an OP_SENDFILEREAD case and seeing if it shows any additional bugs. Mike "Silby" Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
@#$@#$ crap. I think I found a dirty-mmap edge case with truncation. It requires a change to vm_page_set_validclean(), which of course is one of the core routines in the VM system. Basically what happens is that ftruncate() calls vnode_pager_setsize() which eventually calls vm_page_set_validclean(). If you happened to mmap() the truncation point shared R+W and dirty it, then truncate to something that isn't a multiple DEV_BSIZE.. for example, if you were to truncate to an offset of '10', and a buffer has not been instantiated or marked dirty for the block yet, then the truncate operation will clear the dirty bit on the page and your 10 bytes of dirty data will never get synced and will disappear if the page is freed. vm_page_set_validclean() needs to set the valid bits and clear the dirty bits associated with (base,size) within the page. If base and/or size is unaligned then the valid and dirty bits encompass the bits associated with any overlapping DEV_BSIZEd chunks. This is fine for setting valid, but not correct when clearing dirty. Only dirty bits for DEV_BSIZE chunks that are fully enclosed in the range can be cleared. The fix is easy, but a little scary due to being right smack in the middle of the VM system. -- In anycase, I think I got it licked. I'm going to run this nfs tester program overnight on a local filesystem, NFSv2, and NFSv3 mount. Cross your fingers! If it survives I'll start comitting to -current tomorrow. I give it about a 70% chance of surviving. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
> Thanks! I'm slowly whacking the bugs. I just fixed another one... That's awesome... I'd hoped this program might help you find a few things, but I never expected you to find so many bugs in NFS so... quickly! I certainly didn't expect you to tickle any local filesystem problems either. :) > I think I can make it perfect. I'll post another patch tomorrow. Thanks. With 4.5 imminent these improvements are, to say state the flagrantly obvious, very timely indeed. - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
: : Very cool. Good job! : :-DG : :David Greenman :Co-founder, The FreeBSD Project - http://www.freebsd.org Thanks! I'm slowly whacking the bugs. I just fixed another one... vtruncbuf() handles the buffers beyond the file EOF but doesn't handle the buffer straddling the truncation point, so I had to augment the NFS client's truncation code to deal with that. With that fixed the tester program got to 34483 operations before finding a problem. Hopefully I'm in the home stretch now :-) What I really love about this program is that the problems are so repeatable. So far the same failure occurs at exactly the same place, every time. It makes it unbelievably easy to track the bugs down. I think I can make it perfect. I'll post another patch tomorrow. -Matt Matthew Dillon <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
On Wed, 12 Dec 2001, Matthew Dillon wrote: > :I suppose while were on the topic.. > : > :Are there any hidden secrets to eeking out more performance from the BSD > :NFS client (other than version types and the normal fstab tweaks). > : > :Im the CS Labs manager at NetApp..and Im always trying to store away a > :secret here or there when someone comes to me with a problem in the field. > : > :FreeBSD since v2..rock on! > > For extreme performance there are some zero-copy patches floating around > which have not been integrated into the main tree. Generally, though, > your NFS performance is going to be ultimately limited by your server's > disk performance. > > -Matt > Matthew Dillon And if you hadn't heard, Matt just fixed a couple of bugs in the tcp stack which improves NFS greatly. It sounds like after this round of NFS fixes, the first answer to NFS questions should be: Upgrade to 4.5! Mike "Silby" Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
:I suppose while were on the topic.. : :Are there any hidden secrets to eeking out more performance from the BSD :NFS client (other than version types and the normal fstab tweaks). : :Im the CS Labs manager at NetApp..and Im always trying to store away a :secret here or there when someone comes to me with a problem in the field. : :FreeBSD since v2..rock on! * Make sure you don't have packet loss in your network (test with larger packets, aka ping -s 8192 rather then just ping, and perhaps test with a pattern (-p)). * Run a sufficient number of nfsd's on the server side, depending on load. 4 or 8 is typical. * Run nfsiod's on the client side. I usually run 4. This will drastically improve read-ahead and, for example, can bump linear read speeds on a 100BaseTX network from 7 MBytes/sec to 11 MBytes/sec (full saturation). * Use NFS version 3 when possible (this is the default) * Sometimes playing around with the various attribute cache timeouts (see 'man mount_nfs') helps. Sometimes it doesn't. For extreme performance there are some zero-copy patches floating around which have not been integrated into the main tree. Generally, though, your NFS performance is going to be ultimately limited by your server's disk performance. -Matt Matthew Dillon <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
Very cool. Good job! -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
I suppose while were on the topic.. Are there any hidden secrets to eeking out more performance from the BSD NFS client (other than version types and the normal fstab tweaks). Im the CS Labs manager at NetApp..and Im always trying to store away a secret here or there when someone comes to me with a problem in the field. FreeBSD since v2..rock on! To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
Ok, here is the latest patch for -stable. Note that Kirk comitted a
slightly modified version of the softupdates fix to -current already
(the VOP_FSYNC stuff), which I will be MFCing in 3 days.
This still doesn't fix all the problems the nfstest program that Jordan
posted finds, but it sure runs a hellofalot longer now before reporting
an error. 10,000+ tests now before failing (NFSv2 and NFSv3).
Bugs fixed:
* Possible SMP database corruption due to vm_pager_unmap_page()
not clearing the TLB for the other cpu's.
* When flusing a dirty buffer due to B_CACHE getting cleared,
we were accidently setting B_CACHE again (that is, bwrite() sets
B_CACHE), when we really want it to stay clear after the write
is complete. This resulted in a corrupt buffer.
* We have to call vtruncbuf() when ftruncate()ing to remove
any buffer cache buffers. This is still tentitive, I may
be able to remove it due to the second bug fix.
* vnode_pager_setsize() race against nfs_vinvalbuf()... we have
to set n_size before calling nfs_vinvalbuf or the NFS code
may recursively vnode_pager_setsize() to the original value
before the truncate. This is what was causing the user mmap
bus faults in the nfs tester program.
* Fix to softupdates (old version)
There are some general comments in there too. After I do more tests
and cleanups (maybe remove the vtruncbuf()) I will port it all to
-current, test, and commit. So far the stuff is simple enough that
a 3-day MFC will probably suffice.
All I can say is... holy shit!
-Matt
Index: kern/vfs_bio.c
===
RCS file: /home/ncvs/src/sys/kern/vfs_bio.c,v
retrieving revision 1.242.2.13
diff -u -r1.242.2.13 vfs_bio.c
--- kern/vfs_bio.c 2001/11/18 07:10:59 1.242.2.13
+++ kern/vfs_bio.c 2001/12/13 05:52:55
@@ -2189,9 +2189,22 @@
* to softupdates re-dirtying the buffer. In the latter
* case, B_CACHE is set after the first write completes,
* preventing further loops.
+*
+* NOTE! b*write() sets B_CACHE. If we cleared B_CACHE
+* above while extending the buffer, we cannot allow the
+* buffer to remain with B_CACHE set after the write
+* completes or it will represent a corrupt state. To
+* deal with this we set B_NOCACHE to scrap the buffer
+* after the write.
+*
+* We might be able to do something fancy, like setting
+* B_CACHE in bwrite() except if B_DELWRI is already set,
+* so the below call doesn't set B_CACHE, but that gets real
+* confusing. This is much easier.
*/
if ((bp->b_flags & (B_CACHE|B_DELWRI)) == B_DELWRI) {
+ bp->b_flags |= B_NOCACHE;
VOP_BWRITE(bp->b_vp, bp);
goto loop;
}
Index: nfs/nfs_bio.c
===
RCS file: /home/ncvs/src/sys/nfs/Attic/nfs_bio.c,v
retrieving revision 1.83.2.1
diff -u -r1.83.2.1 nfs_bio.c
--- nfs/nfs_bio.c 2001/11/18 07:10:59 1.83.2.1
+++ nfs/nfs_bio.c 2001/12/13 05:52:10
@@ -193,8 +193,14 @@
vm_page_set_validclean(m, 0, size - toff);
/* handled by vm_fault now*/
/* vm_page_zero_invalid(m, TRUE); */
+ } else {
+ /*
+* Read operation was short. If no error occured
+* we may have hit a zero-fill section. We simply
+* leave valid set to 0.
+*/
+ ;
}
-
if (i != ap->a_reqpage) {
/*
* Whether or not to leave the page activated is up in
@@ -896,14 +902,12 @@
else
bcount = np->n_size - (off_t)lbn * biosize;
}
-
- bp = nfs_getcacheblk(vp, lbn, bcount, p);
-
if (uio->uio_offset + n > np->n_size) {
np->n_size = uio->uio_offset + n;
np->n_flag |= NMODIFIED;
vnode_pager_setsize(vp, np->n_size);
}
+ bp = nfs_getcacheblk(vp, lbn, bcount, p);
}
if (!bp) {
@@ -1409,11 +1413,13 @@
io.iov_len = uiop->uio_resid = bp->b_bcount;
io.iov_base = bp->b_data;
uiop
Re: NFS: How to make FreeBSD fall on its face in one easy step
> To be clear, what exactly are you doing? > > It sounds like you're exporting something from freebsd, mounting it on OSX > and running this tool on OSX against the filesystem exported from freebsd ? > > If so, What mount options? NFSv2 or v3? That is correct. As to the NFS options used, I honestly couldn't say since I'm getting at the filesystem through Netinfo and that's handled by OS X's automount daemon, that having no relation whatsoever to AMD and hence no amd.conf file or anything else I can easily look at to determine how it's being mounted. Maybe Mike knows more about how to find this out - he's not in management. :) - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
I found a second bug... nfs truncation code race.
I've enclosed both patches below. NFS truncation race first, softupdates
bug second. The patches are against -stable.
There are still more bugs... the nfstest code is seeing data corruption
on read. It looks like another truncation bug. I'm tracking it down.
-Matt
Index: nfs/nfs_vnops.c
===
RCS file: /home/ncvs/src/sys/nfs/Attic/nfs_vnops.c,v
retrieving revision 1.150.2.4
diff -u -r1.150.2.4 nfs_vnops.c
--- nfs/nfs_vnops.c 2001/08/05 00:23:58 1.150.2.4
+++ nfs/nfs_vnops.c 2001/12/13 01:42:15
@@ -710,6 +710,15 @@
*/
if (vp->v_mount->mnt_flag & MNT_RDONLY)
return (EROFS);
+
+ /*
+* We run vnode_pager_setsize() early (why?),
+* we must set np->n_size now to avoid vinvalbuf
+* V_SAVE races that might setsize a lower
+* value.
+*/
+ tsize = np->n_size;
+ np->n_size = vap->va_size;
vnode_pager_setsize(vp, vap->va_size);
if (np->n_flag & NMODIFIED) {
if (vap->va_size == 0)
@@ -719,12 +728,12 @@
error = nfs_vinvalbuf(vp, V_SAVE,
ap->a_cred, ap->a_p, 1);
if (error) {
+ np->n_size = tsize;
vnode_pager_setsize(vp, np->n_size);
return (error);
}
}
- tsize = np->n_size;
- np->n_size = np->n_vattr.va_size = vap->va_size;
+ np->n_vattr.va_size = vap->va_size;
};
} else if ((vap->va_mtime.tv_sec != VNOVAL ||
vap->va_atime.tv_sec != VNOVAL) && (np->n_flag & NMODIFIED) &&
Index: ufs/ffs/ffs_inode.c
===
RCS file: /home/ncvs/src/sys/ufs/ffs/ffs_inode.c,v
retrieving revision 1.56.2.3
diff -u -r1.56.2.3 ffs_inode.c
--- ufs/ffs/ffs_inode.c 2001/11/20 20:27:27 1.56.2.3
+++ ufs/ffs/ffs_inode.c 2001/12/12 23:43:36
@@ -244,6 +244,10 @@
if (error) {
return (error);
}
+ if (length > 0 && DOINGSOFTDEP(ovp)) {
+ if ((error = VOP_FSYNC(ovp, cred, MNT_WAIT, p)) != 0)
+ return (error);
+ }
oip->i_size = length;
size = blksize(fs, oip, lbn);
if (ovp->v_type != VDIR)
@@ -359,6 +363,10 @@
if (newspace == 0)
panic("ffs_truncate: newspace");
if (oldspace - newspace > 0) {
+ /*
+* XXX Softupdates, adjust queued directblock
+* in place rather then the second FSYNC above?
+*/
/*
* Block number of space to be free'd is
* the old block # plus the number of frags
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
Jordan Hubbard wrote: > > I should point out that FSX can be used against any filesystem, and > > that there are reports locally (at Apple) that it's great for killing > > FreeBSD machines. I wasn't able to reproduce this when I tried, but I > > may not have let it run long enough. > > Oh, it blows freebsd.apple.com right out of the water with a kernel > panic after running for just 3 seconds from an OS X box on the same > LAN segment. :) To be clear, what exactly are you doing? It sounds like you're exporting something from freebsd, mounting it on OSX and running this tool on OSX against the filesystem exported from freebsd ? If so, What mount options? NFSv2 or v3? Cheers, -Peter -- Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
: :> I should point out that FSX can be used against any filesystem, and :> that there are reports locally (at Apple) that it's great for killing :> FreeBSD machines. I wasn't able to reproduce this when I tried, but I :> may not have let it run long enough. : :Oh, it blows freebsd.apple.com right out of the water with a kernel :panic after running for just 3 seconds from an OS X box on the same :LAN segment. :) : :- Jordan What panic are you getting? -Matt Matthew Dillon <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
> I should point out that FSX can be used against any filesystem, and > that there are reports locally (at Apple) that it's great for killing > FreeBSD machines. I wasn't able to reproduce this when I tried, but I > may not have let it run long enough. Oh, it blows freebsd.apple.com right out of the water with a kernel panic after running for just 3 seconds from an OS X box on the same LAN segment. :) - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
: : :I should point out that FSX can be used against any filesystem, and :that there are reports locally (at Apple) that it's great for killing :FreeBSD machines. I wasn't able to reproduce this when I tried, but I :may not have let it run long enough. Well, I already found and tracked down a softupdates bug revealed by this code... definite server panic, especially if an NFSv2 mount is used. I shot some mail off to Kirk with a proposed fix for that. With both NFSv2 and NFSv3 I get user-process bus errors with the truncate()/mmap() combinations being used. I'm tracking that down now. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
I should point out that FSX can be used against any filesystem, and that there are reports locally (at Apple) that it's great for killing FreeBSD machines. I wasn't able to reproduce this when I tried, but I may not have let it run long enough. > Oooh. Very cool! I'll start messing with it (oops, that's going to > make both Paul and Alfred annoyed with me!) 8) > :I'm also trying to determine which of the fixes Apple has made to NFS might > :be adapted to FreeBSD, something which is made more difficult by the fact > :that much of the code was taken straight from 4.4 Lite some time back and > :both operating systems have diverged significantly since then. Many of the key issues in making OS X NFS work were related to its interaction with the UBC and the subtly different VFS semantics, although the same issues probably exist in different form in the FreeBSD code. I get dragged into some really shocking corridor discussions every now and then. 8) = Mike To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
> The only thing I get is a math exception because "closeprob" is zero > since no -c option was given. > > Can you provide some sample parameters please ? Hmmm, how strange, now that I look at the code it's obvious that a divide by zero will occur with a zero closeprob and the docs state the default to be "infinity", which is obviously not the case. The strange part is that I ran this on freebsd.apple.com, which is running 4.4-stable, with one parameter (the filename) exactly as I pasted in the usage instructions before. Perhaps all this time spent living next to the Macintosh in my office has induced that copy of FreeBSD to be more "friendly" and mask simple math errors. :-) In any case, -c 1 appears to work just fine. - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
In message <[EMAIL PROTECTED]>, Jordan Hubbard writ es: >Usage: > cc fsx.c -o fsx > ./fsx /some/nfs/mounted/scratchfile > [ ** kaboom! ** ] The only thing I get is a math exception because "closeprob" is zero since no -c option was given. Can you provide some sample parameters please ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
In message <[EMAIL PROTECTED]> Matthew Dillon writes: : Oooh. Very cool! I'll start messing with it (oops, that's going to : make both Paul and Alfred annoyed with me!) Well, I can understand you not wanting to upset Paul, but anything that annoys Alfred can't be all bad, can it :-) Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: NFS: How to make FreeBSD fall on its face in one easy step
Oooh. Very cool! I'll start messing with it (oops, that's going to make both Paul and Alfred annoyed with me!) -Matt :It came up in a meeting today at Apple just how fragile the BSD NFS :implementation was before significant work was put in to stabilizing it, :and in that discussion came up a little test tool written originally by :Avie Tevanian and subsequently improved by one of the folks here. : :This tool basically tries to do everything it can (legally) to confuse an :NFS server. It seeks around, does I/O to and truncates/changes the size :of a test file, all while doing everything it can to detect data corruption :or other signs of misbehavior which might result from out-of-order replies :or any other previously-observed NFS pathology. Very few NFS implementations :apparently survive this test and FreeBSD's is no exception. The sources are :provided below, courtesy of Avie, for the education and enjoyment(?) of :anyone who's motivated to play with (or even pretends to understand) NFS. : :Usage: : cc fsx.c -o fsx : ./fsx /some/nfs/mounted/scratchfile : [ ** kaboom! ** ] : :I'm also trying to determine which of the fixes Apple has made to NFS might :be adapted to FreeBSD, something which is made more difficult by the fact :that much of the code was taken straight from 4.4 Lite some time back and :both operating systems have diverged significantly since then. Anyone :really keen on investigating this further themselves on it can also go to :http://www.opensource.apple.com/projects/darwin and register themselves :online (it's easy) to access the Darwin CVS repository, the module in :question being "xnu" (the Darwin kernel). Thanks. : :- Jordan : :... To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
NFS: How to make FreeBSD fall on its face in one easy step
It came up in a meeting today at Apple just how fragile the BSD NFS
implementation was before significant work was put in to stabilizing it,
and in that discussion came up a little test tool written originally by
Avie Tevanian and subsequently improved by one of the folks here.
This tool basically tries to do everything it can (legally) to confuse an
NFS server. It seeks around, does I/O to and truncates/changes the size
of a test file, all while doing everything it can to detect data corruption
or other signs of misbehavior which might result from out-of-order replies
or any other previously-observed NFS pathology. Very few NFS implementations
apparently survive this test and FreeBSD's is no exception. The sources are
provided below, courtesy of Avie, for the education and enjoyment(?) of
anyone who's motivated to play with (or even pretends to understand) NFS.
Usage:
cc fsx.c -o fsx
./fsx /some/nfs/mounted/scratchfile
[ ** kaboom! ** ]
I'm also trying to determine which of the fixes Apple has made to NFS might
be adapted to FreeBSD, something which is made more difficult by the fact
that much of the code was taken straight from 4.4 Lite some time back and
both operating systems have diverged significantly since then. Anyone
really keen on investigating this further themselves on it can also go to
http://www.opensource.apple.com/projects/darwin and register themselves
online (it's easy) to access the Darwin CVS repository, the module in
question being "xnu" (the Darwin kernel). Thanks.
- Jordan
/*
* Copyright (C) 1991, NeXT Computer, Inc. All Rights Reserverd.
*
* File: fsx.c
* Author: Avadis Tevanian, Jr.
*
* File system exerciser.
*
* Rewritten 8/98 by Conrad Minshall.
*/
#include
#include
#ifdef _UWIN
# include
# include
# include
# include
# define MAP_FILE 0
#else
# include
#endif
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#define NUMPRINTCOLUMNS 32 /* # columns of data to print on each line */
/*
* A log entry is an operation and a bunch of arguments.
*/
struct log_entry {
int operation;
int args[3];
};
#define LOGSIZE 1000
struct log_entryoplog[LOGSIZE]; /* the log */
int logptr = 0; /* current position in log */
int logcount = 0; /* total ops */
/*
* Define operations
*/
#define OP_READ 1
#define OP_WRITE2
#define OP_TRUNCATE 3
#define OP_CLOSEOPEN4
#define OP_MAPREAD 5
#define OP_MAPWRITE 6
#define OP_SKIPPED 7
#ifndef PAGE_SIZE
#define PAGE_SIZE 4096
#endif
#define PAGE_MASK (PAGE_SIZE - 1)
char*original_buf; /* a pointer to the original data */
char*good_buf; /* a pointer to the correct data */
char*temp_buf; /* a pointer to the current data */
char*fname; /* name of our test file */
int fd; /* fd for our test file */
off_t file_size = 0;
off_t biggest = 0;
charstate[256];
unsigned long testcalls = 0; /* calls to function "test" */
unsigned long simulatedopcount = 0; /* -b flag */
int closeprob = 0; /* -c flag */
int debug = 0; /* -d flag */
unsigned long debugstart = 0; /* -D flag */
unsigned long maxfilelen = 256 * 1024;/* -l flag */
int sizechecks = 1; /* -n flag disables them */
int maxoplen = 64 * 1024; /* -o flag */
int quiet = 0; /* -q flag */
unsigned long progressinterval = 0; /* -p flag */
int readbdy = 1;/* -r flag */
int style = 0; /* -s flag */
int truncbdy = 1; /* -t flag */
int writebdy = 1; /* -w flag */
longmonitorstart = -1; /* -m flag */
longmonitorend = -1;/* -m flag */
int lite = 0; /* -L flag */
longnumops = -1;/* -N flag */
int randomoplen = 1;/* -O flag disables it */
int seed = 1; /* -S flag */
int mapped_writes = 1; /* -W flag disables */
int mapped_reads = 1; /* -R flag disables it */
int fsxgoodfd = 0;
FILE * fsxlogf = NULL;
int badoff = -1;
int closeopen = 0;
void
prt(char *fmt, ...)
{
va_list args;
va_start(args, fmt);
vfprintf(stdout, fmt, args);
if (fsxlogf)
vfprintf(fsxlogf, fmt, args);
va_end(args);
}
void
prterr(char *prefix)
{
prt("%s%s%s\n", prefix, prefix ? ": " : "", strerror(errno));
}
void
log4(int operation, int arg0, int arg1, int arg2)
{
struct log_entry *le;
le = &oplog[logptr];
le->operation =
