Re: Voxer using FreeBSD, BSDNow.tv interview
On Mon, Oct 20, 2014 at 07:18:45PM -0400, Philip M. Gollucci wrote: Not true, you can roll your own omnibus chef builds with this fixed. Can we get this off advocacy@ please? pgp6sj87TQSAd.pgp Description: PGP signature
Re: kernel page fault with nfs
The system now has an uptime of 24h using NFS heavily. So wsize/rsize=2^15-1 seems to have been the problem which is imho a bug therefore. mfg Tobias 2014-10-21 5:11 GMT+02:00 Marcelo Araujo araujobsdp...@gmail.com: Hello Tibias, Any news? Best Regards, 2014-10-20 20:55 GMT+08:00 Rick Macklem rmack...@uoguelph.ca: Tobias C. Berner wrote: Now that I posted it, 32767 should of course be 2^15=32768. Let me recheck if it still hangs with the correct value. On Monday 20 October 2014 09.15:39 Tobias C. Berner wrote: Hi Marcelo Yes, I'm using readahead: The mountoptions are readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late If you type nfsstat -m, you will see what is actually getting used. (I suspect the above rsize/wsize got clipped to 32256 or something like that. I think it clips it to a multiple of 512.) If rsize/wsize are not a power of 2, there are issues, although I've never been able to see why it is broken. Maybe it should clip it to the power of 2 below the value, since it causes unexplained problems otherwise. rick mfg Tobias On Monday 20 October 2014 10.41:30 Marcelo Araujo wrote: Hello Tobias, Could you show how you are mount the NFS share? Are you using 'readahead' option? Best Regards, 2014-10-19 17:40 GMT+08:00 Tobias C. Berner tcber...@gmail.com: both are at 1100038. On Sunday 19 October 2014 11.12:36 Marcelo Araujo wrote: It is still strange, could you do what Allan said and send us the result in case you are not sure you have world and kernel in the same revision! On Oct 19, 2014 6:48 AM, Tobias C. Berner tcber...@gmail.com wrote: Hi World ist from october 16, installed world and kernel then. Kernel was later rebuilt with debug-options. Is the following more sensible? ## # kgdb NOXON/kernel.debug vmcore.1 Fatal trap 12: page fault while in kernel mode cpuid = 5; apic id = 05 fault virtual address = 0xfe07d1744000 fault code = supervisor write data, page not present instruction pointer = 0x20:0x80d4d58a stack pointer = 0x28:0xfe086057f240 frame pointer = 0x28:0xfe086057f2f0 code segment = base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 6524 (python2.7) (kgdb) bt #0 doadump (textdump=1) at pcpu.h:219 #1 0x80926b6d in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:447 #2 0x809270c0 in panic (fmt=value optimized out) at /usr/src/sys/kern/kern_shutdown.c:746 #3 0x8035f167 in db_panic (addr=value optimized out, have_addr=2, count=0, modif=0x0) at /usr/src/sys/ddb/db_command.c:473 #4 0x8035ed7d in db_command (cmd_table=0x0) at /usr/src/sys/ddb/db_command.c:440 #5 0x8035eaf4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:493 #6 0x80361600 in db_trap (type=value optimized out, code=0) at /usr/src/sys/ddb/db_main.c:251 #7 0x80966f01 in kdb_trap (type=12, code=0, tf=value optimized out) at /usr/src/sys/kern/subr_kdb.c:654 #8 0x80d4fa7c in trap_fatal (frame=0xfe086057f190, eva=value optimized out) at /usr/src/sys/amd64/amd64/trap.c:861 #9 0x80d4fe0c in trap_pfault (frame=0xfe086057f190, usermode=value optimized out) at /usr/src/sys/amd64/amd64/trap.c:677 #10 0x80d4f42e in trap (frame=0xfe086057f190) at /usr/src/sys/amd64/amd64/trap.c:426 #11 0x80d33972 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:231 #12 0x80d4d58a in bzero () at /usr/src/sys/amd64/amd64/support.S:53 #13 0x80830463 in ncl_doio (vp=0xf801e7f99938, bp=0xfe07c5a168e8, cr=value optimized out, td=value optimized out, called_from_strategy=value optimized out) at /usr/src/sys/fs/nfsclient/nfs_clbio.c:1648 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org -- -- Marcelo Araujo(__)ara...@freebsd.org
Re: kernel page fault with nfs
Hello Tobias, That sounds good, at least you don't have any crash so far. I agree with you, seems a bug, I'm gonna take a look on that. Could you share with me your testbed or how you can reproduce the issue? Best Regards, 2014-10-21 15:36 GMT+08:00 T.C.Berner tcber...@gmail.com: The system now has an uptime of 24h using NFS heavily. So wsize/rsize=2^15-1 seems to have been the problem which is imho a bug therefore. mfg Tobias 2014-10-21 5:11 GMT+02:00 Marcelo Araujo araujobsdp...@gmail.com: Hello Tibias, Any news? Best Regards, 2014-10-20 20:55 GMT+08:00 Rick Macklem rmack...@uoguelph.ca: Tobias C. Berner wrote: Now that I posted it, 32767 should of course be 2^15=32768. Let me recheck if it still hangs with the correct value. On Monday 20 October 2014 09.15:39 Tobias C. Berner wrote: Hi Marcelo Yes, I'm using readahead: The mountoptions are readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late If you type nfsstat -m, you will see what is actually getting used. (I suspect the above rsize/wsize got clipped to 32256 or something like that. I think it clips it to a multiple of 512.) If rsize/wsize are not a power of 2, there are issues, although I've never been able to see why it is broken. Maybe it should clip it to the power of 2 below the value, since it causes unexplained problems otherwise. rick mfg Tobias On Monday 20 October 2014 10.41:30 Marcelo Araujo wrote: Hello Tobias, Could you show how you are mount the NFS share? Are you using 'readahead' option? Best Regards, 2014-10-19 17:40 GMT+08:00 Tobias C. Berner tcber...@gmail.com: both are at 1100038. On Sunday 19 October 2014 11.12:36 Marcelo Araujo wrote: It is still strange, could you do what Allan said and send us the result in case you are not sure you have world and kernel in the same revision! On Oct 19, 2014 6:48 AM, Tobias C. Berner tcber...@gmail.com wrote: Hi World ist from october 16, installed world and kernel then. Kernel was later rebuilt with debug-options. Is the following more sensible? ## # kgdb NOXON/kernel.debug vmcore.1 Fatal trap 12: page fault while in kernel mode cpuid = 5; apic id = 05 fault virtual address = 0xfe07d1744000 fault code = supervisor write data, page not present instruction pointer = 0x20:0x80d4d58a stack pointer = 0x28:0xfe086057f240 frame pointer = 0x28:0xfe086057f2f0 code segment = base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 6524 (python2.7) (kgdb) bt #0 doadump (textdump=1) at pcpu.h:219 #1 0x80926b6d in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:447 #2 0x809270c0 in panic (fmt=value optimized out) at /usr/src/sys/kern/kern_shutdown.c:746 #3 0x8035f167 in db_panic (addr=value optimized out, have_addr=2, count=0, modif=0x0) at /usr/src/sys/ddb/db_command.c:473 #4 0x8035ed7d in db_command (cmd_table=0x0) at /usr/src/sys/ddb/db_command.c:440 #5 0x8035eaf4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:493 #6 0x80361600 in db_trap (type=value optimized out, code=0) at /usr/src/sys/ddb/db_main.c:251 #7 0x80966f01 in kdb_trap (type=12, code=0, tf=value optimized out) at /usr/src/sys/kern/subr_kdb.c:654 #8 0x80d4fa7c in trap_fatal (frame=0xfe086057f190, eva=value optimized out) at /usr/src/sys/amd64/amd64/trap.c:861 #9 0x80d4fe0c in trap_pfault (frame=0xfe086057f190, usermode=value optimized out) at /usr/src/sys/amd64/amd64/trap.c:677 #10 0x80d4f42e in trap (frame=0xfe086057f190) at /usr/src/sys/amd64/amd64/trap.c:426 #11 0x80d33972 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:231 #12 0x80d4d58a in bzero () at /usr/src/sys/amd64/amd64/support.S:53 #13 0x80830463 in ncl_doio (vp=0xf801e7f99938, bp=0xfe07c5a168e8, cr=value optimized out, td=value optimized out, called_from_strategy=value optimized out) at /usr/src/sys/fs/nfsclient/nfs_clbio.c:1648
Re: HOWTO articles for migrating from Linux to FreeBSD, especially for pkg?
On 21 Oct 2014, at 00:15, Mason Loring Bliss ma...@blisses.org wrote: The second thing that would be useful would be a series of cheat sheets for various things. This can either be equivalent commands or equivalent systems. Let new folks know that LUKS is GELI and that md-raid1 is gmirror and so forth. Show common package handling commands for various Linux flavours and map them to pkgng and ports. For instance, what's the equivalent of yum provides? Or what do I do in place of apt-cache search or zypper up or similar. I agree that this would be useful, but it requires someone familiar with both systems to write. Perhaps you could help by coming up with a list of things that you did frequently with Debian and a description of what they did, then someone more familiar with the FreeBSD side can help fill in any gaps where you haven't yet worked out what the FreeBSD equivalent is (or, if there isn't a FreeBSD equivalent, then we have a useful feature request). Other things in the grab bag... It's generally said that ports and pkgs shouldn't mix, but there are at least a couple instances where it's unavoidable: I bet roughly no one who installs Subversion wants the FreeBSD bug report headers baked in by default, but there they are unless you rebuild from ports with a non-default configuration. It's worth noting that the FreeBSD headers don't affect operation. Subversion only adds the headers to the commit message if they're modified. I think that the fix for this is to add a line at the top saying # Things below this line are only included if modified I find that I do occasionally use those in other projects. If you want to watch DVDs on your FreeBSD workstation, it's necessary to install libdvdcss, but you can't get it from pkgng because it's not there. Again, you must build from ports. Really? I've installed vlc from packages and it seems able to play DVDs. I don't remember having to do anything special. Perhaps I had an old version of libdecss installed. I thought that CSS had been ruled not to be an 'effective copyright protection mechanism' in the US, so wasn't covered by the DMCA anymore. I have nothing against ports, but people are warned off of mixing packages and ports when clearly it's necessary sometimes. Oh, here's one. I *was* horrified by ports at first, until someone told me about make config-recursive. It really makes me wonder why this isn't the default. I remember giving up on FreeBSD when 9.x was new because I had to build X from ports after the FreeBSD breach, and it seemed like the process was going to take a couple days of stuttering stops and starts as random packages I didn't want in some cases popped up between compiles. I learned some mechanism for saying just take the defaults but what I know now is that what I really wanted was make config-recursive. Why, out of curiosity, is it not the default? That would seem better than documenting it harder. The recommended way of building packages now is to use Poudriere. The Poudriere section in the handbook is still very new and contributions are *very* welcome. I think that having an example of 'how to build libdecss from ports' there might be a good idea. There is a plan that each package set should come with a package containing the matching ports tree, so that you can build package from ports that are compatible with the binary ones. That should make a lot of this easier. I think the two cases for Poudriere that need to be in the handbook are: - How to build a few custom packages but mostly use upstream for an individual - How to build a local package repository for your organisation with a load of custom config options for packages built from ports and some custom local-only ports Ah, and one more for the grab bag. I strongly suspect that many folks coming from Linux are going to bristle at the notion of using Sendmail. I used to run it so I wasn't terribly bothered by it, but maybe pre-populating rc.conf with obvious bits that people can see and turn off would be nice. OpenBSD has a nice model of populating rc.conf and sysctl.conf fully, and it ends up being a pleasant tool. Those awash in wonder, coming from Linux, can say, Look, it's all right here! We put those things in /etc/defaults/rc.conf, which makes merges easier on upgrade: the user doesn't touch /etc/defaults/rc.conf and the update tool doesn't touch /etc/rc.conf. Again, if the handbook doesn't tell you to look in /etc/defaults/rc.conf then that's an oversight that we should fix. It might be a good idea to move this thread to the -docs mailing list, as it seems to have identified a number of shortcomings in our documentation and it would be a good idea to try to find some docs people willing to help get them fixed. David ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe,
Re: kernel page fault with nfs
Hi Marcelo The following ist the current fstab-line which seems to run smoothly: odo.firefly:/storage/multimedia /multimedia nfs readahead=4,soft,intr,rw,tcp,wsize=32768,rsize=32768,late 0 0 nfsstat -m: odo.firefly:/storage/multimedia on /multimedia nfsv3,tcp,resvport,soft,intr,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,a cregmax=60,nametimeo=60,negnametimeo=60,rsize=32768,wsize=32768,readdirsi ze=32768,readahead=4,wcommitsize=2798255,timeout=120,retrans=2 Now the bad line (no different appart from the typo) odo.firefly:/storage/multimedia /multimedia nfs readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late 0 0 which leads to the page-faults. And as you said wsize/rsize gets rounded down to the multiple of 512: odo.firefly:/storage/multimedia on /multimedia nfsv3,tcp,resvport,soft,intr,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,a cregmax=60,nametimeo=60,negnametimeo=60,rsize=32256,wsize=32256,readdirsi ze=32256,readahead=4,wcommitsize=2798255,timeout=120,retrans=2 I can easily reproduce the pagefault by letting for example multimedia/gpodder write to the nfs. hope this helps, mfg Tobias On Tuesday 21 October 2014 15.45:24 Marcelo Araujo wrote: Hello Tobias, That sounds good, at least you don't have any crash so far. I agree with you, seems a bug, I'm gonna take a look on that. Could you share with me your testbed or how you can reproduce the issue? Best Regards, 2014-10-21 15:36 GMT+08:00 T.C.Berner tcber...@gmail.com: The system now has an uptime of 24h using NFS heavily. So wsize/rsize=2^15-1 seems to have been the problem which is imho a bug therefore. mfg Tobias 2014-10-21 5:11 GMT+02:00 Marcelo Araujo araujobsdp...@gmail.com: Hello Tibias, Any news? Best Regards, 2014-10-20 20:55 GMT+08:00 Rick Macklem rmack...@uoguelph.ca: Tobias C. Berner wrote: Now that I posted it, 32767 should of course be 2^15=32768. Let me recheck if it still hangs with the correct value. On Monday 20 October 2014 09.15:39 Tobias C. Berner wrote: Hi Marcelo Yes, I'm using readahead: The mountoptions are readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late If you type nfsstat -m, you will see what is actually getting used. (I suspect the above rsize/wsize got clipped to 32256 or something like that. I think it clips it to a multiple of 512.) If rsize/wsize are not a power of 2, there are issues, although I've never been able to see why it is broken. Maybe it should clip it to the power of 2 below the value, since it causes unexplained problems otherwise. rick mfg Tobias On Monday 20 October 2014 10.41:30 Marcelo Araujo wrote: Hello Tobias, Could you show how you are mount the NFS share? Are you using 'readahead' option? Best Regards, 2014-10-19 17:40 GMT+08:00 Tobias C. Berner tcber...@gmail.com: both are at 1100038. On Sunday 19 October 2014 11.12:36 Marcelo Araujo wrote: It is still strange, could you do what Allan said and send us the result in case you are not sure you have world and kernel in the same revision! On Oct 19, 2014 6:48 AM, Tobias C. Berner tcber...@gmail.com wrote: Hi World ist from october 16, installed world and kernel then. Kernel was later rebuilt with debug-options. Is the following more sensible? ## # kgdb NOXON/kernel.debug vmcore.1 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kernel page fault with nfs
Tobias, Thank you very much, it really helps to simulate the problem. I'm gonna try as soon as possible and I will keep you informed. Best Regards, 2014-10-21 17:10 GMT+08:00 Tobias C. Berner tcber...@gmail.com: Hi Marcelo The following ist the current fstab-line which seems to run smoothly: odo.firefly:/storage/multimedia /multimedia nfs readahead=4,soft,intr,rw,tcp,wsize=32768,rsize=32768,late 0 0 nfsstat -m: odo.firefly:/storage/multimedia on /multimedia nfsv3,tcp,resvport,soft,intr,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=32768,wsize=32768,readdirsize=32768,readahead=4,wcommitsize=2798255,timeout=120,retrans=2 Now the bad line (no different appart from the typo) odo.firefly:/storage/multimedia /multimedia nfs readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late 0 0 which leads to the page-faults. And as you said wsize/rsize gets rounded down to the multiple of 512: odo.firefly:/storage/multimedia on /multimedia nfsv3,tcp,resvport,soft,intr,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=32256,wsize=32256,readdirsize=32256,readahead=4,wcommitsize=2798255,timeout=120,retrans=2 I can easily reproduce the pagefault by letting for example multimedia/gpodder write to the nfs. hope this helps, mfg Tobias On Tuesday 21 October 2014 15.45:24 Marcelo Araujo wrote: Hello Tobias, That sounds good, at least you don't have any crash so far. I agree with you, seems a bug, I'm gonna take a look on that. Could you share with me your testbed or how you can reproduce the issue? Best Regards, 2014-10-21 15:36 GMT+08:00 T.C.Berner tcber...@gmail.com: The system now has an uptime of 24h using NFS heavily. So wsize/rsize=2^15-1 seems to have been the problem which is imho a bug therefore. mfg Tobias 2014-10-21 5:11 GMT+02:00 Marcelo Araujo araujobsdp...@gmail.com: Hello Tibias, Any news? Best Regards, 2014-10-20 20:55 GMT+08:00 Rick Macklem rmack...@uoguelph.ca: Tobias C. Berner wrote: Now that I posted it, 32767 should of course be 2^15=32768. Let me recheck if it still hangs with the correct value. On Monday 20 October 2014 09.15:39 Tobias C. Berner wrote: Hi Marcelo Yes, I'm using readahead: The mountoptions are readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late If you type nfsstat -m, you will see what is actually getting used. (I suspect the above rsize/wsize got clipped to 32256 or something like that. I think it clips it to a multiple of 512.) If rsize/wsize are not a power of 2, there are issues, although I've never been able to see why it is broken. Maybe it should clip it to the power of 2 below the value, since it causes unexplained problems otherwise. rick mfg Tobias On Monday 20 October 2014 10.41:30 Marcelo Araujo wrote: Hello Tobias, Could you show how you are mount the NFS share? Are you using 'readahead' option? Best Regards, 2014-10-19 17:40 GMT+08:00 Tobias C. Berner tcber...@gmail.com: both are at 1100038. On Sunday 19 October 2014 11.12:36 Marcelo Araujo wrote: It is still strange, could you do what Allan said and send us the result in case you are not sure you have world and kernel in the same revision! On Oct 19, 2014 6:48 AM, Tobias C. Berner tcber...@gmail.com wrote: Hi World ist from october 16, installed world and kernel then. Kernel was later rebuilt with debug-options. Is the following more sensible? ## # kgdb NOXON/kernel.debug vmcore.1 Fatal trap 12: page fault while in kernel mode cpuid = 5; apic id = 05 fault virtual address = 0xfe07d1744000 fault code = supervisor write data, page not present instruction pointer = 0x20:0x80d4d58a stack pointer = 0x28:0xfe086057f240 frame pointer = 0x28:0xfe086057f2f0 code segment = base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 6524 (python2.7) (kgdb) bt #0 doadump (textdump=1) at pcpu.h:219 #1 0x80926b6d in kern_reboot
Re: kernel page fault with nfs
No problem, I hope you are successful. mfg Tobias On Tuesday 21 October 2014 17.13:32 Marcelo Araujo wrote: Tobias, Thank you very much, it really helps to simulate the problem. I'm gonna try as soon as possible and I will keep you informed. Best Regards, 2014-10-21 17:10 GMT+08:00 Tobias C. Berner tcber...@gmail.com: Hi Marcelo The following ist the current fstab-line which seems to run smoothly: odo.firefly:/storage/multimedia /multimedia nfs readahead=4,soft,intr,rw,tcp,wsize=32768,rsize=32768,late 0 0 nfsstat -m: odo.firefly:/storage/multimedia on /multimedia nfsv3,tcp,resvport,soft,intr,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acre gmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=32768,wsize=32768,re addirsize=32768,readahead=4,wcommitsize=2798255,timeout=120,retrans=2 Now the bad line (no different appart from the typo) odo.firefly:/storage/multimedia /multimedia nfs readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late 0 0 which leads to the page-faults. And as you said wsize/rsize gets rounded down to the multiple of 512: odo.firefly:/storage/multimedia on /multimedia nfsv3,tcp,resvport,soft,intr,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acre gmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=32256,wsize=32256,re addirsize=32256,readahead=4,wcommitsize=2798255,timeout=120,retrans=2 I can easily reproduce the pagefault by letting for example multimedia/gpodder write to the nfs. hope this helps, mfg Tobias On Tuesday 21 October 2014 15.45:24 Marcelo Araujo wrote: Hello Tobias, That sounds good, at least you don't have any crash so far. I agree with you, seems a bug, I'm gonna take a look on that. Could you share with me your testbed or how you can reproduce the issue? Best Regards, 2014-10-21 15:36 GMT+08:00 T.C.Berner tcber...@gmail.com: The system now has an uptime of 24h using NFS heavily. So wsize/rsize=2^15-1 seems to have been the problem which is imho a bug therefore. mfg Tobias 2014-10-21 5:11 GMT+02:00 Marcelo Araujo araujobsdp...@gmail.com: Hello Tibias, Any news? Best Regards, 2014-10-20 20:55 GMT+08:00 Rick Macklem rmack...@uoguelph.ca: Tobias C. Berner wrote: Now that I posted it, 32767 should of course be 2^15=32768. Let me recheck if it still hangs with the correct value. On Monday 20 October 2014 09.15:39 Tobias C. Berner wrote: Hi Marcelo Yes, I'm using readahead: The mountoptions are readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late If you type nfsstat -m, you will see what is actually getting used. (I suspect the above rsize/wsize got clipped to 32256 or something like that. I think it clips it to a multiple of 512.) If rsize/wsize are not a power of 2, there are issues, although I've never been able to see why it is broken. Maybe it should clip it to the power of 2 below the value, since it causes unexplained problems otherwise. rick mfg Tobias On Monday 20 October 2014 10.41:30 Marcelo Araujo wrote: Hello Tobias, Could you show how you are mount the NFS share? Are you using 'readahead' option? Best Regards, 2014-10-19 17:40 GMT+08:00 Tobias C. Berner tcber...@gmail.com: both are at 1100038. On Sunday 19 October 2014 11.12:36 Marcelo Araujo wrote: It is still strange, could you do what Allan said and send us the result in case you are not sure you have world and kernel in the same revision! On Oct 19, 2014 6:48 AM, Tobias C. Berner tcber...@gmail.com wrote: Hi World ist from october 16, installed world and kernel then. Kernel was later rebuilt with debug-options. Is the following more sensible? ## # kgdb NOXON/kernel.debug vmcore.1 Fatal trap 12: page fault while in kernel mode cpuid = 5; apic id = 05 fault virtual address = 0xfe07d1744000
Re: kernel page fault with nfs
Marcelo Araujo wrote: Tobias, Thank you very much, it really helps to simulate the problem. I'm gonna try as soon as possible and I will keep you informed. You are more than welcome to try and find this. However, I don't think folks need to use a non-power of 2 rsize/wsize, so I think I'll change the client to clip rsize/wsize to that. (This example was just a typo.) Is there anyone out there that thinks having an rsize/wsize that isn't a power of 2 is needed? Thanks, rick Best Regards, 2014-10-21 17:10 GMT+08:00 Tobias C. Berner tcber...@gmail.com: Hi Marcelo The following ist the current fstab-line which seems to run smoothly: odo.firefly:/storage/multimedia /multimedia nfs readahead=4,soft,intr,rw,tcp,wsize=32768,rsize=32768,late 0 0 nfsstat -m: odo.firefly:/storage/multimedia on /multimedia nfsv3,tcp,resvport,soft,intr,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=32768,wsize=32768,readdirsize=32768,readahead=4,wcommitsize=2798255,timeout=120,retrans=2 Now the bad line (no different appart from the typo) odo.firefly:/storage/multimedia /multimedia nfs readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late 0 0 which leads to the page-faults. And as you said wsize/rsize gets rounded down to the multiple of 512: odo.firefly:/storage/multimedia on /multimedia nfsv3,tcp,resvport,soft,intr,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=32256,wsize=32256,readdirsize=32256,readahead=4,wcommitsize=2798255,timeout=120,retrans=2 I can easily reproduce the pagefault by letting for example multimedia/gpodder write to the nfs. hope this helps, mfg Tobias On Tuesday 21 October 2014 15.45:24 Marcelo Araujo wrote: Hello Tobias, That sounds good, at least you don't have any crash so far. I agree with you, seems a bug, I'm gonna take a look on that. Could you share with me your testbed or how you can reproduce the issue? Best Regards, 2014-10-21 15:36 GMT+08:00 T.C.Berner tcber...@gmail.com: The system now has an uptime of 24h using NFS heavily. So wsize/rsize=2^15-1 seems to have been the problem which is imho a bug therefore. mfg Tobias 2014-10-21 5:11 GMT+02:00 Marcelo Araujo araujobsdp...@gmail.com: Hello Tibias, Any news? Best Regards, 2014-10-20 20:55 GMT+08:00 Rick Macklem rmack...@uoguelph.ca: Tobias C. Berner wrote: Now that I posted it, 32767 should of course be 2^15=32768. Let me recheck if it still hangs with the correct value. On Monday 20 October 2014 09.15:39 Tobias C. Berner wrote: Hi Marcelo Yes, I'm using readahead: The mountoptions are readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late If you type nfsstat -m, you will see what is actually getting used. (I suspect the above rsize/wsize got clipped to 32256 or something like that. I think it clips it to a multiple of 512.) If rsize/wsize are not a power of 2, there are issues, although I've never been able to see why it is broken. Maybe it should clip it to the power of 2 below the value, since it causes unexplained problems otherwise. rick mfg Tobias On Monday 20 October 2014 10.41:30 Marcelo Araujo wrote: Hello Tobias, Could you show how you are mount the NFS share? Are you using 'readahead' option? Best Regards, 2014-10-19 17:40 GMT+08:00 Tobias C. Berner tcber...@gmail.com: both are at 1100038. On Sunday 19 October 2014 11.12:36 Marcelo Araujo wrote: It is still strange, could you do what Allan said and send us the result in case you are not sure you have world and kernel in the same revision! On Oct 19, 2014 6:48 AM, Tobias C. Berner tcber...@gmail.com wrote: Hi World ist from october 16, installed world and kernel then. Kernel was later rebuilt with debug-options. Is the following more sensible? ## # kgdb NOXON/kernel.debug vmcore.1 Fatal trap 12: page fault while in kernel mode cpuid = 5; apic id = 05 fault virtual address = 0xfe07d1744000
Re: kernel page fault with nfs
Marcelo Araujo wrote: Tobias, Thank you very much, it really helps to simulate the problem. I'm gonna try as soon as possible and I will keep you informed. Oh, and since the NFS client code is straightforward, I think the bug is somewhere in the buffer cache/vm code. Maybe for cases where the size of the buffer in the buffer cache isn't an exact multiple of PAGE_SIZE. rick Best Regards, 2014-10-21 17:10 GMT+08:00 Tobias C. Berner tcber...@gmail.com: Hi Marcelo The following ist the current fstab-line which seems to run smoothly: odo.firefly:/storage/multimedia /multimedia nfs readahead=4,soft,intr,rw,tcp,wsize=32768,rsize=32768,late 0 0 nfsstat -m: odo.firefly:/storage/multimedia on /multimedia nfsv3,tcp,resvport,soft,intr,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=32768,wsize=32768,readdirsize=32768,readahead=4,wcommitsize=2798255,timeout=120,retrans=2 Now the bad line (no different appart from the typo) odo.firefly:/storage/multimedia /multimedia nfs readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late 0 0 which leads to the page-faults. And as you said wsize/rsize gets rounded down to the multiple of 512: odo.firefly:/storage/multimedia on /multimedia nfsv3,tcp,resvport,soft,intr,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=32256,wsize=32256,readdirsize=32256,readahead=4,wcommitsize=2798255,timeout=120,retrans=2 I can easily reproduce the pagefault by letting for example multimedia/gpodder write to the nfs. hope this helps, mfg Tobias On Tuesday 21 October 2014 15.45:24 Marcelo Araujo wrote: Hello Tobias, That sounds good, at least you don't have any crash so far. I agree with you, seems a bug, I'm gonna take a look on that. Could you share with me your testbed or how you can reproduce the issue? Best Regards, 2014-10-21 15:36 GMT+08:00 T.C.Berner tcber...@gmail.com: The system now has an uptime of 24h using NFS heavily. So wsize/rsize=2^15-1 seems to have been the problem which is imho a bug therefore. mfg Tobias 2014-10-21 5:11 GMT+02:00 Marcelo Araujo araujobsdp...@gmail.com: Hello Tibias, Any news? Best Regards, 2014-10-20 20:55 GMT+08:00 Rick Macklem rmack...@uoguelph.ca: Tobias C. Berner wrote: Now that I posted it, 32767 should of course be 2^15=32768. Let me recheck if it still hangs with the correct value. On Monday 20 October 2014 09.15:39 Tobias C. Berner wrote: Hi Marcelo Yes, I'm using readahead: The mountoptions are readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late If you type nfsstat -m, you will see what is actually getting used. (I suspect the above rsize/wsize got clipped to 32256 or something like that. I think it clips it to a multiple of 512.) If rsize/wsize are not a power of 2, there are issues, although I've never been able to see why it is broken. Maybe it should clip it to the power of 2 below the value, since it causes unexplained problems otherwise. rick mfg Tobias On Monday 20 October 2014 10.41:30 Marcelo Araujo wrote: Hello Tobias, Could you show how you are mount the NFS share? Are you using 'readahead' option? Best Regards, 2014-10-19 17:40 GMT+08:00 Tobias C. Berner tcber...@gmail.com: both are at 1100038. On Sunday 19 October 2014 11.12:36 Marcelo Araujo wrote: It is still strange, could you do what Allan said and send us the result in case you are not sure you have world and kernel in the same revision! On Oct 19, 2014 6:48 AM, Tobias C. Berner tcber...@gmail.com wrote: Hi World ist from october 16, installed world and kernel then. Kernel was later rebuilt with debug-options. Is the following more sensible? ## # kgdb NOXON/kernel.debug vmcore.1 Fatal trap 12: page fault while in kernel mode cpuid = 5; apic id = 05 fault virtual address = 0xfe07d1744000 fault code = supervisor write data, page not present
Re: HOWTO articles for migrating from Linux to FreeBSD, especially for pkg?
On Tue, Oct 21, 2014 at 09:14:07AM +0100, David Chisnall wrote: I agree that this would be useful, but it requires someone familiar with both systems to write. Doing this stuff professionally, I could probably come up with equivalences for a number of Unices. What would be a reasonable path to getting write privs on the wiki? I bet roughly no one who installs Subversion wants the FreeBSD bug report headers baked in by default, It's worth noting that the FreeBSD headers don't affect operation. It mostly violates the principle of least surprise and is a cosmetic blemish. I'd suspect that a lot of people use Subversion for their own or their company development, and the default behaviour looks strange. It was certainly surprising to me in any event. A default of not having that turned on but an option to turn it on seems like the most reasonable thing, given that the option is so closely tied to FreeBSD development. It might be a good idea to move this thread to the -docs mailing list, I will subscribe to that. -- Mason Loring Bliss (( If I have not seen as far as others, it is because ma...@blisses.org )) giants were standing on my shoulders. - Hal Abelson ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
ZFS: i/o error - all block copies unavailable #2
Hi! I saw posts from 2009 on this mailing list, about a boot issue with ZFS. I have IBM server with HW RAID10 made from 36 disks. I'm using FreeBSD 9.2-RELEASE. From system point of view my raid is seen as 65TiB drive at /dev/mfid0 From ZFS point of view it's plain stripe as one zvol (on mfid0p2). My partition layout is GPT: root@backup7:~ # gpart show = 34 140624990141 mfid0 GPT (65T) 34 128 1 freebsd-boot (64k) 162 140624990013 2 freebsd-zfs (65T) It was installed using mfsbsd installer based on FreeBSD 9.2. This machine has been working happily, until today when I did reboot. After that system is unbootable, and throws on boot: ZFS: i/o error - all block copies unavailable ZFS: can't find root dsl_dir My pool is ok, I can boot to it through external ISO (f.e. mfsbsd). Also status shows that everything's fine (screenshot of what my kvm shows there): http://s.verknowsys.com/5f6de0dfbc6d576dc53c000409a27e344e60adee.png I saw in previous thread (http://marc.info/?l=freebsd-fsm=125807564213035w=2), that there's a patch for raidz that somewhat fixes that issue in FreeBSD 8.x with raidz. So far I did try many solutions but none helped in that matter. I did try: update bootcode, move /boot from one place to other, update kernel, switch between kernel / kernel.old, and several more or less stupid attempts to fix it Any ideas? -- Daniel (dmilith) Dettlaff ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Build failed in Jenkins: FreeBSD_HEAD #1671
See https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/1671/changes Changes: [ngie] Port t_chroot to FreeBSD - Add missing #include sys/stat.h for mkdir(2) - Omit the fchroot(2) tests because the support is not present on FreeBSD Sponsored by: EMC / Isilon Storage Division [ngie] unlink(/) fails with EISDIR instead of EBUSY on FreeBSD; test for that instead Sponsored by: EMC / Isilon Storage Division [ngie] Mark osi __unused so this compiles cleanly on FreeBSD Sponsored by: EMC / Isilon Storage Division [jimharris] ixl: remove i40e_register_x710_int.h This file is not used by the FreeBSD ixl driver. Submitted by: Eric Joyner eric.joy...@intel.com MFC after: 3 days [ngie] Port t_write to FreeBSD - Mark the signo variable for the signal handle __unused - Use limits.h instead of sys/syslimits.h (the latter does not exist on FreeBSD) Sponsored by: EMC / Isilon Storage Division [jmg] it is not cast to a pointer of the specified type, it is cast to the specified type... mtod(m, uint8_t) does not work, mtod(m, uint8_t *) does work.. [ngie] Add missing #include for sys/stat.h for fchmod Sponsored by: EMC / Isilon Storage Division [ngie] libutil.h is required for fparseln on FreeBSD Sponsored by: EMC / Isilon Storage Division [ngie] Port lib/libc/gen/t_siginfo to FreeBSD - mcontext_t on FreeBSD doesn't have a __gregs field (it's split out on FreeBSD into separate fields). In order to avoid muddying the test code with MD code, the debugging trace info has not been implemented - FreeBSD does not implement the si_stime and si_utime fields in siginfo_t, so omit the debugging code that dumps the values - sys/inttypes.h doesn't exist on FreeBSD Sponsored by: EMC / Isilon Storage Division [jmg] spell out the arguments.. the + *offsetp does not belong w/ the type, move it outside the .Fn macro... [bapt] Fix build by marking the new functions as weak This is a temporary fix [bapt] Add support for __cxa_throw_bad_array_new_length in libcxxrt It is required for use with newer g++49 Differential Revision: https://reviews.freebsd.org/D982 Reviewed by:theraven Approved by:theraven MFC after: 3 weeks [br] Add driver for Micrel KSZ9021 Gigabit Ethernet Transceiver (PHY). Sponsored by: DARPA, AFRL [hselasky] Fix minor typo in currently unused macro. MFC after: 3 days [hselasky] Fix multiple incorrect SYSCTL arguments in the kernel: - Wrong integer type was specified. - Wrong or missing access specifier. The access specifier sometimes included the SYSCTL type, which it should not, except for procedural SYSCTL nodes. - Logical OR where binary OR was expected. - Properly assert the access argument passed to all SYSCTL macros, using the CTASSERT macro. This applies to both static- and dynamically created SYSCTLs. - Properly assert the the data type for both static and dynamic SYSCTLs. In the case of static SYSCTLs we only assert that the data pointed to by the SYSCTL data pointer has the correct size, hence there is no easy way to assert types in the C language outside a C-function. - Rewrote some code which doesn't pass a constant access specifier when creating dynamic SYSCTL nodes, which is now a requirement. - Updated EXAMPLES section in SYSCTL manual page. MFC after: 3 days Sponsored by: Mellanox Technologies [kevlo] Add the Intel BayTrail USB device which needs port routing for USB 3.0. Tested on the BayTrail E3845 platform. Reviewed by:hselasky [neel] Merge projects/bhyve_svm into HEAD. After this change bhyve supports AMD processors with the SVM/AMD-V hardware extensions. More details available here: https://lists.freebsd.org/pipermail/freebsd-virtualization/2014-October/002905.html Submitted by: Anish Gupta (akgu...@gmail.com) Tested by: Benjamin Perrault (ben.perra...@gmail.com) Tested by: Willem Jan Withagen (w...@digiware.nl) [bryanv] Use the size of the Ethernet address, not the entire header, when copying into forwarding entry. Reported by:Coverity CID:1248849 [markj] Correct the calculation of tcps_rto in the struct tcpcb - tcpsinfo_t translator. Submitted by: Grenville Armitage garmit...@swin.edu.au MFC after: 1 week [markj] Fix a few small bugs in the DTrace USDT rules: * anchor search strings appropriately, * use .ALLSRC to pass the full path to the D script to dtrace(1), * don't insert the auto-generated header into SRCS - it doesn't accomplish anything, and we end up having to remove it from OBJS anyway. Reviewed by:rpaulo Differential Revision: https://reviews.freebsd.org/D978 MFC after: 3 weeks Sponsored by: EMC / Isilon Storage Division -- [...truncated 131915 lines...] gzip -cn https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/lib/libgssapi/gss_accept_sec_context.3 gss_accept_sec_context.3.gz --- gss_acquire_cred.3.gz --- gzip -cn
ZFS: i/o error - all block copies unavailable #2
Hi! I saw posts from 2009 on this mailing list, about a boot issue with ZFS. I have IBM server with HW RAID10 made from 36 disks. I'm using FreeBSD 9.2-RELEASE. From system point of view my raid is seen as 65TiB drive at /dev/mfid0 From ZFS point of view it's plain stripe as one zvol (on mfid0p2). My partition layout is GPT: root@backup7:~ # gpart show = 34 140624990141 mfid0 GPT (65T) 34 128 1 freebsd-boot (64k) 162 140624990013 2 freebsd-zfs (65T) It was installed using mfsbsd installer based on FreeBSD 9.2. This machine has been working happily, until today when I did reboot. After that system is unbootable, and throws on boot: ZFS: i/o error - all block copies unavailable ZFS: can't find root dsl_dir My pool is ok, I can boot to it through external ISO (f.e. mfsbsd). Also status shows that everything's fine (screenshot of what my kvm shows there): http://s.verknowsys.com/5f6de0dfbc6d576dc53c000409a27e344e60adee.png I saw in previous thread (http://marc.info/?l=freebsd-fsm=125807564213035w=2), that there's a patch for raidz that somewhat fixes that issue in FreeBSD 8.x with raidz. So far I did try many solutions but none helped in that matter. I did try: update bootcode, move /boot from one place to other, update kernel, switch between kernel / kernel.old, and several more or less stupid attempts to fix it Any ideas? -- Daniel (dmilith) Dettlaff irc: dmilith on irc.freenode.net skype: dmilith ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
SVN r273381/2 breaks world
From an empty obj tree on the second pass through, I get .. --- tblgen --- c++ -O2 -pipe -I/usr/src/usr.bin/clang/tblgen/../../../contrib/llvm/include -I/usr/src/usr.bin/clang/tblgen/../../../contrib/llvm/tools/clang/include -I/usr/src/usr.bin/clang/tblgen/../../../contrib/llvm/utils/TableGen -I. -I/usr/src/usr.bin/clang/tblgen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=\x86_64-unknown-freebsd11.0\ -DLLVM_HOST_TRIPLE=\x86_64-unknown-freebsd11.0\ -DDEFAULT_SYSROOT=\\ -Qunused-arguments -I/usr/obj/usr/src/tmp/legacy/usr/include -fno-exceptions -fno-rtti -Wno-c++11-extensions -static -L/usr/obj/usr/src/tmp/legacy/usr/lib -o tblgen AsmMatcherEmitter.o AsmWriterEmitter.o AsmWriterInst.o CTagsEmitter.o CallingConvEmitter.o CodeEmitterGen.o CodeGenDAGPatterns.o CodeGenInstruction.o CodeGenMapTable.o CodeGenRegisters.o CodeGenSchedule.o CodeGenTarget.o DAGISelEmitter.o DAGISelMatcher.o DAGISelMatcherEmitter.o DAGISelMatcherGen.o DAGISelMatcherOpt.o DFAPacketizerEmitter.o DisassemblerEmitter.o FastISelEmitter.o FixedLenDecoderEmitter.o InstrInfoEmitter.o IntrinsicEmitter.o OptParserEmitter.o PseudoLoweringEmitter.o RegisterInfoEmitter.o SetTheory.o SubtargetEmitter.o TGValueTypes.o TableGen.o X86DisassemblerTables.o X86ModRMFilters.o X86RecognizableInstr.o /usr/obj/usr/src/tmp/usr/src/usr.bin/clang/tblgen/../../../lib/clang/libllvmtablegen/libllvmtablegen.a /usr/obj/usr/src/tmp/usr/src/usr.bin/clang/tblgen/../../../lib/clang/libllvmsupport/libllvmsupport.a -lncursesw -legacy /usr/lib/libc++.a(cxxrt_stdexcept.o):(.rodata+0x1c0): multiple definition of `typeinfo for std::bad_array_new_length' /usr/lib/libc++.a(new.o):(.rodata+0x90): first defined here /usr/lib/libc++.a(cxxrt_stdexcept.o):(.rodata+0x1a0): multiple definition of `typeinfo name for std::bad_array_new_length' /usr/lib/libc++.a(new.o):(.rodata+0x70): first defined here /usr/lib/libc++.a(cxxrt_stdexcept.o):(.rodata+0xc0): multiple definition of `vtable for std::bad_array_new_length' /usr/lib/libc++.a(new.o):(.rodata+0x10): first defined here c++: error: linker command failed with exit code 1 (use -v to see invocation) *** [tblgen] Error code 1 make[3]: stopped in /usr/src/usr.bin/clang/tblgen 1 error Recompiling and reinstalling libc++ doesn't change anything, imb ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfs recv hangs in kmem arena
On Sun, Oct 19, 2014 at 10:30 AM, James R. Van Artsdalen james-freebsd-...@jrv.org wrote: Removing kern.maxfiles from loader.conf still hangs in kmem arena. I tried using a memstick image of -CURRENT made from the release/ process and this also hangs in kmem arena An uninvolved server of mine hung Friday night in statekmem arena during periodic's zpool history. After a reboot it did not hang Saturday night. How up to date is your source tree? r2720221 is relevant. Without that change, there are circumstances in which the code that is supposed to free space from the kmem arena doesn't get called. On 10/16/2014 11:37 PM, James R. Van Artsdalen wrote: On 10/16/2014 11:10 PM, Xin Li wrote: On 10/16/14 8:43 PM, James R. Van Artsdalen wrote: On 10/16/2014 11:12 AM, Xin Li wrote: On 9/26/2014 1:42 AM, James R. Van Artsdalen wrote: FreeBSD BLACKIE.housenet.jrv 10.1-BETA2 FreeBSD 10.1-BETA2 #2 r272070M: Wed Sep 24 17:36:56 CDT 2014 ja...@blackie.housenet.jrv:/usr/obj/usr/src/sys/GENERIC amd64 With current STABLE10 I am unable to replicate a ZFS pool using zfs send/recv without zfs hanging in state kmem arena, within the first 4TB or so (of a 23TB Pool). What does procstat -kk 1176 (or the PID of your 'zfs' process that stuck in that state) say? Cheers, SUPERTEX:/root# ps -lp 866 UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 866 863 0 52 0 66800 29716 kmem are D+1 57:40.82 zfs recv -duvF BIGTOX SUPERTEX:/root# procstat -kk 866 PIDTID COMM TDNAME KSTACK 866 101573 zfs -mi_switch+0xe1 sleepq_wait+0x3a _cv_wait+0x16d vmem_xalloc+0x568 vmem_alloc+0x3d kmem_malloc+0x33 keg_alloc_slab+0xcd keg_fetch_slab+0x151 zone_fetch_slab+0x7e zone_import+0x40 uma_zalloc_arg+0x34e arc_get_data_buf+0x31a arc_buf_alloc+0xaa dmu_buf_will_fill+0x169 dmu_write+0xfc dmu_recv_stream+0xd40 zfs_ioc_recv+0x94e zfsdev_ioctl+0x5ca Do you have any special tuning in your /boot/loader.conf? Cheers, Below. I had forgotten some of this was there. After sending the previous message I ran kgdb to see if I could get a backtrace with function args. I didn't see how to do it for this proc, but during all this the process un-blocked and started running again. The process blocked again in kmem arena after a few minutes. SUPERTEX:/root# cat /boot/loader.conf zfs_load=YES # ZFS vfs.root.mountfrom=zfs:SUPERTEX/UNIX# Specify root partition in a way the # kernel understands kern.maxfiles=32K# Set the sys. wide open files limit kern.ktrace.request_pool=512 #vfs.zfs.debug=1 vfs.zfs.check_hostid=0 loader_logo=beastie# Desired logo: fbsdbw, beastiebw, beastie, none boot_verbose=YES# -v: Causes extra debugging information to be printed geom_mirror_load=YES# RAID1 disk driver (see gmirror(8)) geom_label_load=YES# File system labels (see glabel(8)) ahci_load=YES siis_load=YES mvs_load=YES coretemp_load=YES# Intel Core CPU temperature monitor #console=comconsole kern.msgbufsize=131072# Set size of kernel message buffer kern.geom.label.gpt.enable=0 kern.geom.label.gptid.enable=0 kern.geom.label.disk_ident.enable=0 SUPERTEX:/root# ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfs recv hangs in kmem arena
On Tue, Oct 21, 2014 at 5:03 PM, Alan Cox alan.l@gmail.com wrote: On Sun, Oct 19, 2014 at 10:30 AM, James R. Van Artsdalen james-freebsd-...@jrv.org wrote: Removing kern.maxfiles from loader.conf still hangs in kmem arena. I tried using a memstick image of -CURRENT made from the release/ process and this also hangs in kmem arena An uninvolved server of mine hung Friday night in statekmem arena during periodic's zpool history. After a reboot it did not hang Saturday night. How up to date is your source tree? r2720221 is relevant. Without that change, there are circumstances in which the code that is supposed to free space from the kmem arena doesn't get called. That should be r272071. On 10/16/2014 11:37 PM, James R. Van Artsdalen wrote: On 10/16/2014 11:10 PM, Xin Li wrote: On 10/16/14 8:43 PM, James R. Van Artsdalen wrote: On 10/16/2014 11:12 AM, Xin Li wrote: On 9/26/2014 1:42 AM, James R. Van Artsdalen wrote: FreeBSD BLACKIE.housenet.jrv 10.1-BETA2 FreeBSD 10.1-BETA2 #2 r272070M: Wed Sep 24 17:36:56 CDT 2014 ja...@blackie.housenet.jrv:/usr/obj/usr/src/sys/GENERIC amd64 With current STABLE10 I am unable to replicate a ZFS pool using zfs send/recv without zfs hanging in state kmem arena, within the first 4TB or so (of a 23TB Pool). What does procstat -kk 1176 (or the PID of your 'zfs' process that stuck in that state) say? Cheers, SUPERTEX:/root# ps -lp 866 UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 866 863 0 52 0 66800 29716 kmem are D+1 57:40.82 zfs recv -duvF BIGTOX SUPERTEX:/root# procstat -kk 866 PIDTID COMM TDNAME KSTACK 866 101573 zfs -mi_switch+0xe1 sleepq_wait+0x3a _cv_wait+0x16d vmem_xalloc+0x568 vmem_alloc+0x3d kmem_malloc+0x33 keg_alloc_slab+0xcd keg_fetch_slab+0x151 zone_fetch_slab+0x7e zone_import+0x40 uma_zalloc_arg+0x34e arc_get_data_buf+0x31a arc_buf_alloc+0xaa dmu_buf_will_fill+0x169 dmu_write+0xfc dmu_recv_stream+0xd40 zfs_ioc_recv+0x94e zfsdev_ioctl+0x5ca Do you have any special tuning in your /boot/loader.conf? Cheers, Below. I had forgotten some of this was there. After sending the previous message I ran kgdb to see if I could get a backtrace with function args. I didn't see how to do it for this proc, but during all this the process un-blocked and started running again. The process blocked again in kmem arena after a few minutes. SUPERTEX:/root# cat /boot/loader.conf zfs_load=YES # ZFS vfs.root.mountfrom=zfs:SUPERTEX/UNIX# Specify root partition in a way the # kernel understands kern.maxfiles=32K# Set the sys. wide open files limit kern.ktrace.request_pool=512 #vfs.zfs.debug=1 vfs.zfs.check_hostid=0 loader_logo=beastie# Desired logo: fbsdbw, beastiebw, beastie, none boot_verbose=YES# -v: Causes extra debugging information to be printed geom_mirror_load=YES# RAID1 disk driver (see gmirror(8)) geom_label_load=YES# File system labels (see glabel(8)) ahci_load=YES siis_load=YES mvs_load=YES coretemp_load=YES# Intel Core CPU temperature monitor #console=comconsole kern.msgbufsize=131072# Set size of kernel message buffer kern.geom.label.gpt.enable=0 kern.geom.label.gptid.enable=0 kern.geom.label.disk_ident.enable=0 SUPERTEX:/root# ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SVN r273381/2 breaks world
On Tue, Oct 21, 2014 at 05:34:48PM -0400, Michael Butler wrote: From an empty obj tree on the second pass through, I get .. --- tblgen --- c++ -O2 -pipe -I/usr/src/usr.bin/clang/tblgen/../../../contrib/llvm/include -I/usr/src/usr.bin/clang/tblgen/../../../contrib/llvm/tools/clang/include -I/usr/src/usr.bin/clang/tblgen/../../../contrib/llvm/utils/TableGen -I. -I/usr/src/usr.bin/clang/tblgen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=\x86_64-unknown-freebsd11.0\ -DLLVM_HOST_TRIPLE=\x86_64-unknown-freebsd11.0\ -DDEFAULT_SYSROOT=\\ -Qunused-arguments -I/usr/obj/usr/src/tmp/legacy/usr/include -fno-exceptions -fno-rtti -Wno-c++11-extensions -static -L/usr/obj/usr/src/tmp/legacy/usr/lib -o tblgen AsmMatcherEmitter.o AsmWriterEmitter.o AsmWriterInst.o CTagsEmitter.o CallingConvEmitter.o CodeEmitterGen.o CodeGenDAGPatterns.o CodeGenInstruction.o CodeGenMapTable.o CodeGenRegisters.o CodeGenSchedule.o CodeGenTarget.o DAGISelEmitter.o DAGISelMatcher.o DAGISelMatcherEmitter.o DAGISelMatcherGen.o DAGISelMatcherOpt.o DFAPacketizerEmitter.o DisassemblerEmitter.o FastISelEmitter.o FixedLenDecoderEmitter.o InstrInfoEmitter.o IntrinsicEmitter.o OptParserEmitter.o PseudoLoweringEmitter.o RegisterInfoEmitter.o SetTheory.o SubtargetEmitter.o TGValueTypes.o TableGen.o X86DisassemblerTables.o X86ModRMFilters.o X86RecognizableInstr.o /usr/obj/usr/src/tmp/usr/src/usr.bin/clang/tblgen/../../../lib/clang/libllvmtablegen/libllvmtablegen.a /usr/obj/usr/src/tmp/usr/src/usr.bin/clang/tblgen/../../../lib/clang/libllvmsupport/libllvmsupport.a -lncursesw -legacy /usr/lib/libc++.a(cxxrt_stdexcept.o):(.rodata+0x1c0): multiple definition of `typeinfo for std::bad_array_new_length' /usr/lib/libc++.a(new.o):(.rodata+0x90): first defined here /usr/lib/libc++.a(cxxrt_stdexcept.o):(.rodata+0x1a0): multiple definition of `typeinfo name for std::bad_array_new_length' /usr/lib/libc++.a(new.o):(.rodata+0x70): first defined here /usr/lib/libc++.a(cxxrt_stdexcept.o):(.rodata+0xc0): multiple definition of `vtable for std::bad_array_new_length' /usr/lib/libc++.a(new.o):(.rodata+0x10): first defined here c++: error: linker command failed with exit code 1 (use -v to see invocation) *** [tblgen] Error code 1 make[3]: stopped in /usr/src/usr.bin/clang/tblgen 1 error Recompiling and reinstalling libc++ doesn't change anything, imb ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org Fixed, Bapt pgpjzn0wi77U_.pgp Description: PGP signature
Build failed in Jenkins: FreeBSD_HEAD #1672
See https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/1672/changes Changes: [mjg] tmpfs: allow shared file lookups Tested by: pho [bapt] Revert r273426 r273409 A solution that work with both new and old binutils should be investigated [bapt] older binutils does not know about --no-fatal-warnings [mjg] Mark some more sysctl stuff shared-locked and MPSAFE. [hselasky] Use the UAUTO SYSCTL type for exporting the bounce zone alignment, because the variable size depends on the build type. Reported by:kib @ MFC after: 3 days [emaste] Fix typo in src option description [emaste] Regenerate after r273418 [imp] For the kernel, we have USB_GADGET_EXAMPLES as defaults to yes. For userland defaults to no. This caused issues for the automated option documenation script. Turns out, this isn't used in userland at all, so just remove it from here. [imp] Generate both userland and kernel option settings for showconfig. PR: 191920 [imp] You aren't allowed to test WITH_xxx or WITHOUT_xxx here, so remove it. Even if you were allowed to test for it, the test makes no sense as it always results in adding -DWITH_ATF unless WITH_ATF was already defined. But if MK_ATF != no, then we know it was defined. This, in turn, caused tools/build/options/makemake always think WITH_ATF is the default, which removed control of that from sys.conf.mk. To get the intent of the deleted comment, another mechanism is required, assuming that the intent of that comment is desirable. [ngie] Add sys/socket.h #include for bind(2), et al Sponsored by: EMC / Isilon Storage Division [bapt] Do not make ld(1) warnings fatal anymore, binutils behaviour has changed over the time and gnu.warnings.symbol are now being fatal preventing building world. in the futur we want to investigate only making the gnu.warning.symbol non fatal Reviewed by:imp [bapt] Make the external toolchain support grows to the knowleged of XXFLAGS for C++ dedicated flags and DEPFLAGS for mkdep flags Pass the path to the libc++ headers in both, enforce the gnu++11 standard in the XXFLAGS to satisfy libc++ requirements pass the libc++ objectdir as a location where to find libraries so it can find libstdc++.so and libstdc++.A Reviewed by:imp [bapt] When using an external gcc 4.8+ and not building libstdc++ then create in the objectdir a fake libstdc++.so and libstdc++.a which is a symlink on libc++ that allow g++ to satisfy its links dependencies in the least hackish way. Please note that this hacky libstds++ never get installed on the final system Reviewed by:imp [bapt] Always use libc++ as the default c++ stack when building with an external gcc 4.8+ While here disable building gcc from base when using gcc 4.8+ Reviewed by:imp [bapt] When using an external toolchain note that gcc 4.8+ supports C++11 Submitted by: imp [bapt] The dependencies are computed with CC even if sources are C++, when building when building with an external gcc, we want to be able to pass the path to the libc++ headers so dependencies are correctly computed for C++ source files. Add a DEPFLAGS for that purpose Reviewed by:imp [mjg] Make sysctl name2oid shared-locked as well. This is a follow-up to r273401. [gjb] Fix an issue where a FreeBSD virtual machine provisioned in the Microsoft Azure service does not recognize the second attached disk on the system. Submitted by: kyliel@Microsoft Patched by: weh@Microsoft PR: 194376 MFC after: 3 days X-MFC-10.1: yes, ASAP Sponsored by: The FreeBSD Foundation [mjg] Implement shared locking for sysctl. [mjg] Rename sysctl_lock and _unlock to sysctl_xlock and _xunlock. -- [...truncated 114662 lines...] === bin/csh (all) --- games.all__D --- --- fortune --- cc -O2 -pipe -DDEBUG -Ihttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/games/fortune/fortune/../strfile -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -o fortune fortune.o --- lib.all__D --- --- divdi3.po --- cc -pg -O2 -pipe -fpic -fvisibility=hidden -DVISIBILITY_HIDDEN -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Qunused-arguments -c
Re: zfs recv hangs in kmem arena
On 10/21/2014 5:03 PM, Alan Cox wrote: How up to date is your source tree? r2720221 is relevant. Without that change, there are circumstances in which the code that is supposed to free space from the kmem arena doesn't get called. I've tried HEAD/CURRENT at r272749 On 10-STABLE through r273364 - I do a nightly build test. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kernel page fault with nfs
2014-10-21 20:48 GMT+08:00 Rick Macklem rmack...@uoguelph.ca: Marcelo Araujo wrote: Tobias, Thank you very much, it really helps to simulate the problem. I'm gonna try as soon as possible and I will keep you informed. You are more than welcome to try and find this. However, I don't think folks need to use a non-power of 2 rsize/wsize, so I think I'll change the client to clip rsize/wsize to that. (This example was just a typo.) Yes, that would be nice if you can do that. As seems you gonna take a look on it, I'm gonna move to other things. Is there anyone out there that thinks having an rsize/wsize that isn't a power of 2 is needed? Personally, I can't see where it would be needed. Thanks, rick Best Regards, 2014-10-21 17:10 GMT+08:00 Tobias C. Berner tcber...@gmail.com: Hi Marcelo The following ist the current fstab-line which seems to run smoothly: odo.firefly:/storage/multimedia /multimedia nfs readahead=4,soft,intr,rw,tcp,wsize=32768,rsize=32768,late 0 0 nfsstat -m: odo.firefly:/storage/multimedia on /multimedia nfsv3,tcp,resvport,soft,intr,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=32768,wsize=32768,readdirsize=32768,readahead=4,wcommitsize=2798255,timeout=120,retrans=2 Now the bad line (no different appart from the typo) odo.firefly:/storage/multimedia /multimedia nfs readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late 0 0 which leads to the page-faults. And as you said wsize/rsize gets rounded down to the multiple of 512: odo.firefly:/storage/multimedia on /multimedia nfsv3,tcp,resvport,soft,intr,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=32256,wsize=32256,readdirsize=32256,readahead=4,wcommitsize=2798255,timeout=120,retrans=2 I can easily reproduce the pagefault by letting for example multimedia/gpodder write to the nfs. hope this helps, mfg Tobias On Tuesday 21 October 2014 15.45:24 Marcelo Araujo wrote: Hello Tobias, That sounds good, at least you don't have any crash so far. I agree with you, seems a bug, I'm gonna take a look on that. Could you share with me your testbed or how you can reproduce the issue? Best Regards, 2014-10-21 15:36 GMT+08:00 T.C.Berner tcber...@gmail.com: The system now has an uptime of 24h using NFS heavily. So wsize/rsize=2^15-1 seems to have been the problem which is imho a bug therefore. mfg Tobias 2014-10-21 5:11 GMT+02:00 Marcelo Araujo araujobsdp...@gmail.com: Hello Tibias, Any news? Best Regards, 2014-10-20 20:55 GMT+08:00 Rick Macklem rmack...@uoguelph.ca: Tobias C. Berner wrote: Now that I posted it, 32767 should of course be 2^15=32768. Let me recheck if it still hangs with the correct value. On Monday 20 October 2014 09.15:39 Tobias C. Berner wrote: Hi Marcelo Yes, I'm using readahead: The mountoptions are readahead=4,soft,intr,rw,tcp,wsize=32767,rsize=32767,late If you type nfsstat -m, you will see what is actually getting used. (I suspect the above rsize/wsize got clipped to 32256 or something like that. I think it clips it to a multiple of 512.) If rsize/wsize are not a power of 2, there are issues, although I've never been able to see why it is broken. Maybe it should clip it to the power of 2 below the value, since it causes unexplained problems otherwise. rick mfg Tobias On Monday 20 October 2014 10.41:30 Marcelo Araujo wrote: Hello Tobias, Could you show how you are mount the NFS share? Are you using 'readahead' option? Best Regards, 2014-10-19 17:40 GMT+08:00 Tobias C. Berner tcber...@gmail.com: both are at 1100038. On Sunday 19 October 2014 11.12:36 Marcelo Araujo wrote: It is still strange, could you do what Allan said and send us the result in case you are not sure you have world and kernel in the same revision! On Oct 19, 2014 6:48 AM, Tobias C. Berner tcber...@gmail.com wrote: Hi World ist from october 16, installed world and
Build failed in Jenkins: FreeBSD_HEAD #1673
See https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/1673/changes Changes: [mjg] filedesc: cleanup setugidsafety a little Rename it to fdsetugidsafety for consistency with other functions. There is no need to take filedesc lock if not closing any files. The loop has to verify each file and we are guaranteed fdtable has space for at least 20 fds. As such there is no need to check fd_lastfile. While here tidy up is_unsafe. [mjg] Eliminate unnecessary memory allocation in sys_getgroups and its ibcs2 counterpart. [bapt] Do not define bad_array_new_length::bad_array_new_length in libc++ anymore when used in combinaison with libcxxrt since it is now defined there already. This fixes building world [gjb] Bump __FreeBSD_version to track SA-14:20, SA-14:21, SA-14:22, SA-14:23 Approved by:re (implicit) Sponsored by: The FreeBSD Foundation [mjg] Take the lock shared in linker_search_symbol_name. This helps sysctl kern.proc.stack. -- [...truncated 113853 lines...] --- cmpti2.po --- cc -pg -O2 -pipe -fpic -fvisibility=hidden -DVISIBILITY_HIDDEN -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Qunused-arguments -c https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/lib/libcompiler_rt/../../contrib/compiler-rt/lib/cmpti2.c -o cmpti2.po --- bin.all__D --- --- cp.o --- cc -O2 -pipe -DVM_AND_BUFFER_CACHE_SYNCHRONIZED -D_ACL_PRIVATE -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -c https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/bin/cp/cp.c --- lib.all__D --- --- comparedf2.po --- cc -pg -O2 -pipe -fpic -fvisibility=hidden -DVISIBILITY_HIDDEN -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Qunused-arguments -c https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/lib/libcompiler_rt/../../contrib/compiler-rt/lib/comparedf2.c -o comparedf2.po --- bin.all__D --- --- utils.o --- cc -O2 -pipe -DVM_AND_BUFFER_CACHE_SYNCHRONIZED -D_ACL_PRIVATE -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -c https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/bin/cp/utils.c --- lib.all__D --- --- comparesf2.po --- cc -pg -O2 -pipe -fpic -fvisibility=hidden -DVISIBILITY_HIDDEN -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Qunused-arguments -c https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/lib/libcompiler_rt/../../contrib/compiler-rt/lib/comparesf2.c -o comparesf2.po --- games.all__D --- --- fortune.6.gz --- gzip -cn https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/games/fortune/fortune/fortune.6 fortune.6.gz --- fortune --- cc -O2 -pipe -DDEBUG -Ihttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/games/fortune/fortune/../strfile -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -o fortune fortune.o --- lib.all__D
