Re: [patch] Disable TSC on SMP VMs
The patch I referenced (and forgot to attach) is located here: http://www.silby.com/tsc.c-no_vm_smp_tsc.patch Thanks to those who pointed out my error to me. :) Mike Silby Silbersack On Tue, 16 Aug 2011, Mike Silbersack wrote: Hi all, Thanks to a lightning strike, I recently upgraded my ESXi 4.1 system from an old Athlon X2 to a shiny new Core i5-2500K. I order to see how fast it was, I decided to run some buildworlds. I tried my FreeBSD 9 based VM first, and was unhappy with what I saw - many mpt timeouts were reported, and the filesystem ended up requiring a lot of fscking. My FreeBSD 8 based VM, on the other hand, was quite happy. After some trial and error, I determined that using the TSC as a timecounter with the VM in SMP mode was the root cause. If I switch the VM to one vCPU and run with the TSC, the buildworld runs fine. If I leave it with two vCPUs and change the timecounter to ACPI, then it runs fine too. Based on these results, I created the attached patch; it turns the priority of the TSC down to -100 if it determines that it is running inside a virtualized SMP environment. (I did not do testing on KVM or qemu, but I am assuming that they probably do not have TSC emulation perfect either.) I'd like to get this into 9.0 so that it works reliably for people who run it in a virtual environment on modern hardware. If someone could code review the patch, I would greatly appreciate it. Thanks, Mike Silby Silbersack ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
[patch] Disable TSC on SMP VMs
Hi all, Thanks to a lightning strike, I recently upgraded my ESXi 4.1 system from an old Athlon X2 to a shiny new Core i5-2500K. I order to see how fast it was, I decided to run some buildworlds. I tried my FreeBSD 9 based VM first, and was unhappy with what I saw - many mpt timeouts were reported, and the filesystem ended up requiring a lot of fscking. My FreeBSD 8 based VM, on the other hand, was quite happy. After some trial and error, I determined that using the TSC as a timecounter with the VM in SMP mode was the root cause. If I switch the VM to one vCPU and run with the TSC, the buildworld runs fine. If I leave it with two vCPUs and change the timecounter to ACPI, then it runs fine too. Based on these results, I created the attached patch; it turns the priority of the TSC down to -100 if it determines that it is running inside a virtualized SMP environment. (I did not do testing on KVM or qemu, but I am assuming that they probably do not have TSC emulation perfect either.) I'd like to get this into 9.0 so that it works reliably for people who run it in a virtual environment on modern hardware. If someone could code review the patch, I would greatly appreciate it. Thanks, Mike Silby Silbersack ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: tcp hostcache and ip fastforward for review
On Sun, 9 Nov 2003, Andre Oppermann wrote: Hello all, this patch contains three things (to be separated for committing): I don't have much time free in the next week, so I cannot do a complete review. However, I just did a quick readthrough. tcp_hostcache This looks good to me, I've been waiting for you to finish it for a long time. You actually missed a point: - Ensures that a cached entry isn't added until the 3WHS is completed. This should help make synfloods with random source addresses less damaging. Would it be possible to provide a way for netstat to view the host cache table? I think that it would be useful. ip_fastforward No comment, I didn't read through this part, and I'm not familiar with the forwarding code. tcp bug fixes and MSS DoS attack prevention Generally good, but: - adds tcp_minmssoverload which disconnects a TCP session if it receives too many (1000) packets per second whose average segement size is lower than tcp_minmss - DoS attack 2: make MSS very low on local side of connection and send mny small packet to remote host. For every packet (eg. 2 bytes payload) a sowakeup is done to the listening process. Consumes a lot of CPU there. I don't think that your patch for this really solves anything. Anyone who would write such a program could just as easily make it use concurrent connections, have it auto-reconnect, and/or have it only send 900 packets per second. I think that you should remove this section of the patch, but leave a comment about this problem existing so that it will be thought more about in the future. After the rest of the code is in, we can brainstorm on other possible solutions... I think that Mini's idea of approaching it as an optimization is the correct one. Mike Silby Silbersack ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: tcp hostcache and ip fastforward for review
On Mon, 10 Nov 2003, Andre Oppermann wrote: - Ensures that a cached entry isn't added until the 3WHS is completed. This should help make synfloods with random source addresses less damaging. The cache will only be updated if the tcp connection is being closed. All updates are done in tcp_drop. The T/TCP updates have to be done inline during connection setup. I've converted all places which updated the T/TCP rtmetrics in routing table with updates to the hostcache. Good, that's exactly how it should work. Would it be possible to provide a way for netstat to view the host cache table? I think that it would be useful. At the moment is visible via sysctl -a net.inet.tcp.hostcache.list. Syncache ain't visible via netstat either. So far you had to use route get x.x.x.x to see the rtmetrics for a (host-)route. So I'm sure whether netstat is the right place for it. But I can do that in a second step. Ok, that should be good enough for now. The actually solves the problem. Let me explain in more detail. When we get so many small packets per second the CPU will become pretty saturated. Depending on how much data is sent it can go on for minutes or hours. This code jumps in there and disconnects the within a second. Of course someone can immediatly reconnect and do it again. But that needs the 3WHS again and gives some delay. In the end this code is like the ICMP rate limiter code. It there to migitate a problem to manageable level, not to make it go away. Ok, so the problem is that the sockbuf chain keeps getting longer, causing the delay to grow as more fragments pile in... I see now. I drop my objection to it. Mike Silby Silbersack ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 5.1-p10 reproducible crash with Apache2
On Wed, 5 Nov 2003, [ISO-8859-1] Branko F. Grac(nar wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Silbersack wrote: | Can you try updating to 5.1-current and see if the situation changes at | all? A lot has changed since 5.1-release. If it's still broken in | 5.1-current, we can take a look into it. | | Thanks, I tried today with yesterday's -CURRENT. Same symptoms. No kernel panic, just lockup. Brane Ok, submit a PR with clear details on how to recreate the problem, and we'll see if someone can take a look into it. I'm too busy to look at it, but at least putting it in a PR will ensure that it doesn't get too lost. Once the PR is filed, you might want to try asking on the freebsd-threads list; it sounds like the issue might be thread-related. (Note that your original e-mail might contain enough detail, I'm not certain; I just skimmed it. Filing a good PR is important either way, mailing list messages get easily lost.) Thanks, Mike Silby Silbersack ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 5.1-p10 reproducible crash with Apache2
On Mon, 3 Nov 2003, [ISO-8859-1] Branko F. Grac(nar wrote: Machine locked up in about 5 seconds (this is 1u p4 xeon 2.4 GHz, 2GB of ~ ram). This only accours if Apache2 SSLMutex is set to 'sem' and SSLSessionCache is set to 'shm:/path(size)'. So... there are possible problems with shared memory implementation in 5.1-RELEASE Brane Can you try updating to 5.1-current and see if the situation changes at all? A lot has changed since 5.1-release. If it's still broken in 5.1-current, we can take a look into it. Thanks, Mike Silby Silbersack ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Release Engineering Status Report
On Tue, 16 Sep 2003, Scott Long wrote: Patches have been floated on the mailing list that revert PAE in its various stages. Maybe those need to be brought back up. Silby? Tor? Scott I believe that Tor's commit on August 30th resolved the PAE-related problems, so there is no need for a reversion. Since that time, I've seen three panics posted: 1. Some netinet/ related panic which I couldn't make heads or tails of, and I haven't any followup reports from the poster. 2. Maxim's buildworld -j64 memory kmap entry exhaustion panic, which can be fixed by increasing the number of kmap entries. (Tor has a patch for this, I will probably commit it soon.) 3. A panic caused by sending 64K-1 ping packets, which I can't reproduce. (There's also a small problem with if_xl on pentium-1 machines, but since it's my fault and I'm waiting on test results from a guy, we won't talk about it.) (Hey, anyone have a pentium-200 and a 3com 905B card? Contact me, further testing can't hurt.) So, as far as I can tell, there are no remaining problems related to PAE; I believe that most people are venting frustration that built up between August 9th and 30th. Mike Silby Silbersack ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: new panic during buildworld -j4
Duplicate free from the (32) zone. I'll retype the rest in a few hours when I have time. On Thu, 7 Aug 2003, Bosko Milekic wrote: On Wed, Aug 06, 2003 at 11:13:41PM -0500, Mike Silbersack wrote: I suppose a coredump would be nice here, but I didn't have that enabled... And it turns out that I'm too lazy to actually type in all of the arguments, but I'll leave the machine sitting at the backtrace. If anyone wants any more info, please ask. panic uma_dbg_free uma_zfree_arg free workitem_free free_diradd handle_written_filepage softdep_disk_write_complete bufdone bufdonebio biodone g_dev_done biodone g_io_schedule g_up_procbody fork_exit fork_trampoline --- trap 0x1, eip = 0, esp = 0xd23a5d7c, ebp = 0 --- How about the panic message? There are 4 different panics that could have occured from uma_dbg_free(). Arguments would also be nice, I guess. -- Bosko Milekic * [EMAIL PROTECTED] * [EMAIL PROTECTED] TECHNOkRATIS Consulting Services * http://www.technokratis.com/ ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
new panic during buildworld -j4
I suppose a coredump would be nice here, but I didn't have that enabled... And it turns out that I'm too lazy to actually type in all of the arguments, but I'll leave the machine sitting at the backtrace. If anyone wants any more info, please ask. panic uma_dbg_free uma_zfree_arg free workitem_free free_diradd handle_written_filepage softdep_disk_write_complete bufdone bufdonebio biodone g_dev_done biodone g_io_schedule g_up_procbody fork_exit fork_trampoline --- trap 0x1, eip = 0, esp = 0xd23a5d7c, ebp = 0 --- ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
another buildworld -j4 panic
ok, here goes: duplicate free from zone FFS1 dinode traceback: Debugger panic uma_dbg_free uma_zfree_arg ffs_ifree ufs_reclaim ufs_vnoperate vclean gdonel getnewvnode ffs_vget ufs_lookup ufs_vnoperate vfs_cache_lookup ufs_vnoperate lookup namei stat syscall Xint0x80_syscall Whee! ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: new panic during buildworld -j4
This is from a kernel with sources from soon after i386/pmap.c 1.423 was committed, FWIW. On Wed, 6 Aug 2003, Mike Silbersack wrote: I suppose a coredump would be nice here, but I didn't have that enabled... And it turns out that I'm too lazy to actually type in all of the arguments, but I'll leave the machine sitting at the backtrace. If anyone wants any more info, please ask. panic uma_dbg_free uma_zfree_arg free workitem_free free_diradd handle_written_filepage softdep_disk_write_complete bufdone bufdonebio biodone g_dev_done biodone g_io_schedule g_up_procbody fork_exit fork_trampoline --- trap 0x1, eip = 0, esp = 0xd23a5d7c, ebp = 0 --- ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
show locks w/o ddb?
This may be a dumb question, but is there a way to call whatever show locks calls from within the kernel, similar to how we can call backtrace()? Thanks, Mike Silby Silbersack ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: bug in big pipe code causing performance problems
On Wed, 30 Jul 2003, Pierre Beyssac wrote:
On Wed, Jul 30, 2003 at 11:32:49PM +0200, Pierre Beyssac wrote:
- if (space 0 (wpipe-pipe_buffer.cnt PIPE_SIZE)) {
+ if (space 0
+wpipe-pipe_buffer.cnt wpipe-pipe_buffer.size) {
PS : not-so-obvious after all since the above is equivalent to
(space 0) by itself, so I won't commit the above as is, and the
real fix might be something more complicated...
--
Pierre Beyssac[EMAIL PROTECTED] [EMAIL PROTECTED]
Free domains: http://www.eu.org/ or mail [EMAIL PROTECTED]
Good timing. :)
I was just writing a reply which stated that (space 0) would be
sufficient. However, as you point out, there might be something subtle
lurking around. I'll do some doublechecking tonight and get back to you.
Mike Silby Silbersack
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Best way to get max KVA setting?
On Wed, 25 Jun 2003, Terry Lambert wrote: Mike Silbersack wrote: I could probably grep for this, but what's the best way to get a hold of the # of pages (or MB of ram) that max KVA is set to? I'm adding another autosize option, and I want to base it on min (KVA, ram) so that it doesn't balloon on boxes where ram KVA. I posted a patch about a month ago that did exactly this. It was for a guy who was was running with PAE enabled on an 8G machine, and the autotuning was shooting him in the foot when it tried to grab enough memory to create kmem_map entries for the 8G of RAM in his 2G KVA space, and its head exploded. Let me know if you can't find it in the -current archives, and I can dig it out for you, if you're willing to wait a day or two. -- Terry I found the thread in question, but I didn't see a patch. Mike Silby Silbersack ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Best way to get max KVA setting?
I could probably grep for this, but what's the best way to get a hold of the # of pages (or MB of ram) that max KVA is set to? I'm adding another autosize option, and I want to base it on min (KVA, ram) so that it doesn't balloon on boxes where ram KVA. Thanks, Mike Silby Silbersack ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ACPI: working ACPI vs broken ACPI
On Sat, 15 Feb 2003, Martin Blapp wrote: Feb 13 17:41:05 ibm-01 kernel: ACPI-0625: *** Info: GPE Block0 defined as GPE0 to GPE31 Feb 13 17:41:05 ibm-01 kernel: ACPI-0625: *** Info: GPE Block1 defined as GPE32 to GPE63 I see similar errors on my Presario 2100US... Wild guess: Seem to result from this. If I'm looking at NetBSD's http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/dev/acpi/acpi_ec.c they have a lot done since they took acpi from us. I'm sure it works for netbsd. Is anybody working on this ? Martin I've been trying to load that URL since yesterday, but it's not working from here. Can you elaborate on what it does? Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: -mcpu=pentiumpro still evil?
On Tue, 24 Sep 2002, Alexander Kabaev wrote: On Tue, 24 Sep 2002 10:52:10 -0500 (CDT) Mike Silbersack [EMAIL PROTECTED] wrote: Do you want me to try your first patch? I never got a chance to test it.(And no longer have a copy of it, either.) No, there is a bug in the patch you tested. Could you please try again with an updated patch? URL is the same http://people.freebsd.org/~kan/gcc-cpp.diff -- Alexander Kabaev That doesn't seem to have fixed the problem, and the backtrace looks to be the same: #0 0x08058cb6 in cpp_ideq () #1 0x080592e6 in _cpp_lex_direct () #2 0x08058f6d in _cpp_lex_token () #3 0x08056465 in cpp_macro_definition () #4 0x080564ed in cpp_macro_definition () #5 0x0805672b in _cpp_handle_directive () #6 0x08058f9c in _cpp_lex_token () #7 0x08055832 in cpp_get_token () #8 0x0805595d in cpp_scan_nooutput () #9 0x08048409 in do_preprocessing () #10 0x08048241 in main () #11 0x08048145 in _start () Thanks to the wonderful sort breakage, I'm seeing this if I touch cppmacro.c and make again: === cc_int cc -O -pipe -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX=\/usr\ -I/usr/obj/usr/src/gnu/usr.bin/cc/cc_int/../cc_tools -I/usr/src/gnu/usr.bin/cc/cc_int/../cc_tools -I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc -I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/config -DHAVE_CONFIG_H -DTARGET_NAME=\i386-undermydesk-freebsd\ -DIN_GCC -c /usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/cppmacro.c -o cppmacro.o building static cc_int library sort: open failed: +1: No such file or directory sort: open failed: +1: No such file or directory ranlib libcc_int.a Any chance that's causing a problem? Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: -mcpu=pentiumpro still evil?
On Sun, 22 Sep 2002, Alexander Kabaev wrote: On Sun, 22 Sep 2002 18:51:14 -0500 (CDT) Mike Silbersack [EMAIL PROTECTED] wrote: I'm seeing the segfault in the kernel make depend step, just as someone else reported. OK, could you please try the patch at http://people.freebsd.org/~kan/gcc-cpp.diff and let me know the results. -- Alexander Kabaev This version of the gcc-cpp.diff patch: Index: cppmacro.c === RCS file: /usr/ncvs/src/contrib/gcc/cppmacro.c,v retrieving revision 1.1.1.4 diff -u -r1.1.1.4 cppmacro.c --- cppmacro.c 1 Sep 2002 20:37:29 - 1.1.1.4 +++ cppmacro.c 23 Sep 2002 17:44:32 - @@ -349,6 +349,8 @@ /* Commit the memory, including NUL, and return the token. */ len = dest - BUFF_FRONT (pfile-u_buff); + if ((size_t) (BUFF_LIMIT (pfile-u_buff) - dest) 1) +_cpp_extend_buff (pfile, pfile-u_buff, 1); BUFF_FRONT (pfile-u_buff) = dest + 1; return new_string_token (pfile, dest - len, len); } Does _not_ fix the problem for me. Here's the backtrace of the crash with the patch applied: #0 0x08058cb6 in cpp_ideq () #1 0x080592e6 in _cpp_lex_direct () #2 0x08058f6d in _cpp_lex_token () #3 0x08056465 in cpp_macro_definition () #4 0x080564ed in cpp_macro_definition () #5 0x0805672b in _cpp_handle_directive () #6 0x08058f9c in _cpp_lex_token () #7 0x08055832 in cpp_get_token () #8 0x0805595d in cpp_scan_nooutput () #9 0x08048409 in do_preprocessing () #10 0x08048241 in main () #11 0x08048145 in _start () And of course, it's this part of a buildkernel where it happens: make -V CFILES -V SYSTEM_CFILES -V GEN_CFILES -V GEN_M_CFILES | MKDEP_CPP=cc -E CC=cc xargs mkdep -a -f .newdep -O -pipe -mcpu=pentiumpro -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -fformat-extensions -ansi -g -nostdinc -I- -I. -I/usr/src/sys -I/usr/src/sys/dev -I/usr/src/sys/contrib/dev/acpica -I/usr/src/sys/contrib/ipfilter -D_KERNEL -include opt_global.h -fno-common -mpreferred-stack-boundary=2 -ffreestanding cc: Internal error: Segmentation fault (program cpp0) Please submit a full bug report. See URL:http://www.gnu.org/software/gcc/bugs.html for instructions. Do you want me to try your first patch? I never got a chance to test it. (And no longer have a copy of it, either.) Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: -mcpu=pentiumpro still evil?
On Tue, 24 Sep 2002, David Wolfskill wrote: building static cc_int library sort: open failed: +1: No such file or directory sort: open failed: +1: No such file or directory ranlib libcc_int.a Any chance that's causing a problem? To fix that (regardless of sort), s/sort +1/sort -k 2/ in `which lorder` (and /usr/src/usr.bin/lorder/lorder.sh). Cheers, david (links to my resume at http://www.catwhisker.org/~david) -- David H. Wolfskill[EMAIL PROTECTED] Ok, I fixed lorder.sh, and made gcc again from clean with Alexander's patch. No change, I still see the same segmentation fault. Alexander, how can I easily build gcc with full debugging symbols? That might make the backtrace more useful for you. Or better yet, I'm just including my kernel config. Presumably with it you'll be able to recreate the problem on your system. If not, then we can see what else differs about my system. Mike Silby Silbersack # # GENERIC -- Generic kernel configuration file for FreeBSD/i386 # # For more information on this file, please read the handbook section on # Kernel Configuration Files: # #http://www.FreeBSD.org/handbook/kernelconfig-config.html # # The handbook is also available locally in /usr/share/doc/handbook # if you've installed the doc distribution, otherwise always see the # FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the # latest information. # # An exhaustive list of options and more detailed explanations of the # device lines is also present in the NOTES configuration file. If you are # in doubt as to the purpose or necessity of a line, check first in NOTES. # # $FreeBSD: src/sys/i386/conf/GENERIC,v 1.308 2001/05/13 20:52:39 phk Exp $ machine i386 cpu I486_CPU cpu I586_CPU cpu I686_CPU ident PATROCLES maxusers0 #options RANDOM_IP_ID options UFS_DIRHASH options DIAGNOSTIC #makeoptions NO_WERROR=true device smbus # Bus support, required for smb below. device intpm device alpm device ichsmb device smb device iicbus # Bus support, required for ic/iic/iicsmb below. device iicbb device ic device iic device iicsmb # smb over i2c bridge #device pcf #To statically compile in device wiring instead of /boot/device.hints #hints GENERIC.hints #Default places to look for devices. makeoptions DEBUG=-g#Build kernel with gdb(1) debug symbols options DDB #optionsMATH_EMULATE#Support for x87 emulation options INET#InterNETworking options INET6 #IPv6 communications protocols options FFS #Berkeley Fast Filesystem options SOFTUPDATES #Enable FFS soft updates support options MD_ROOT #MD is a potential root device #optionsNFS #Network Filesystem #optionsNFS_ROOT#NFS usable as root device, NFS required options MSDOSFS #MSDOS Filesystem options CD9660 #ISO 9660 Filesystem options PROCFS #Process filesystem options PSEUDOFS options COMPAT_43 #Compatible with BSD 4.3 [KEEP THIS!] options SCSI_DELAY=15000#Delay (in ms) before probing SCSI options KTRACE #ktrace(1) support options SYSVSHM #SYSV-style shared memory options SYSVMSG #SYSV-style message queues options SYSVSEM #SYSV-style semaphores options P1003_1B#Posix P1003_1B real-time extensions options _KPOSIX_PRIORITY_SCHEDULING options KBD_INSTALL_CDEV# install a CDEV entry in /dev # Debugging for use in -current #optionsDDB #optionsINVARIANTS #optionsINVARIANT_SUPPORT #optionsWITNESS # To make an SMP kernel, the next two are needed #optionsSMP # Symmetric MultiProcessor Kernel #optionsAPIC_IO # Symmetric (APIC) I/O device isa device eisa device pci # Floppy drives device fdc # ATA and ATAPI devices device ata device atadisk # ATA disk drives device atapicd # ATAPI CDROM drives device atapifd # ATAPI floppy drives device atapist # ATAPI tape drives options ATA_STATIC_ID #Static device numbering # SCSI Controllers device ahb # EISA AHA1742 family device ahc # AHA2940 and onboard AIC7xxx devices
Re: -mcpu=pentiumpro still evil?
On Tue, 24 Sep 2002, Craig Rodrigues wrote: On Tue, Sep 24, 2002 at 02:05:08PM -0500, Mike Silbersack wrote: Ok, I fixed lorder.sh, and made gcc again from clean with Alexander's patch. No change, I still see the same segmentation fault. Alexander, how can I easily build gcc with full debugging symbols? That might make the backtrace more useful for you. Does this work for you: cd /usr/src/gnu/usr.bin/cc make DEBUG_FLAGS=-g install -- Craig Rodrigues http://www.gis.net/~craigr [EMAIL PROTECTED] Nope, that doesn't build in debugging symbols either. Neither does adding +g or +gstabs+ to CFLAGS in make.conf. Hmph. Maybe I need to add LDFLAGS or something, I'm not sure. (I'm really bad with Makefiles.) Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: -mcpu=pentiumpro still evil?
On Tue, 24 Sep 2002, Alexander Kabaev wrote: On Tue, 24 Sep 2002 16:07:39 -0500 (CDT) Mike Silbersack [EMAIL PROTECTED] wrote: On Tue, 24 Sep 2002, Craig Rodrigues wrote: On Tue, Sep 24, 2002 at 02:05:08PM -0500, Mike Silbersack wrote: Ok, I fixed lorder.sh, and made gcc again from clean with Alexander's patch. No change, I still see the same segmentation fault. Alexander, how can I easily build gcc with full debugging symbols? That might make the backtrace more useful for you. cd /usr/src/gnu/usr.bin/cc CFLAGS=-g STRIP= make clean all install Always worked for me. -- Alexander Kabaev Yep, STRIP= was the necessary trick, I didn't realize that install -s meant strip. :) As to your patch... it turns out that I wasn't using it. I've been testing with make buildkernel, which uses the copy of gcc built by your last buildworld, not a more recent manual build of gcc. Hence, I've been testing the wrong version. I'll go ahead and run a full buildworld before testing again. Sorry about my poor testing practices. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: -mcpu=pentiumpro still evil?
On Tue, 24 Sep 2002, Mike Silbersack wrote: Yep, STRIP= was the necessary trick, I didn't realize that install -s meant strip. :) As to your patch... it turns out that I wasn't using it. I've been testing with make buildkernel, which uses the copy of gcc built by your last buildworld, not a more recent manual build of gcc. Hence, I've been testing the wrong version. I'll go ahead and run a full buildworld before testing again. Sorry about my poor testing practices. Mike Silby Silbersack Ok, now that I tested properly, I can confirm that your generation 3 patch seems to solve the problem here. Please commit it asap! Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: -mcpu=pentiumpro still evil?
On Sun, 22 Sep 2002, Alexander Kabaev wrote: On Sat, 21 Sep 2002 23:10:51 -0500 (CDT) Mike Silbersack [EMAIL PROTECTED] wrote: Is anyone else still seeing Sig 11's from GCC when mcpu=pentiumpro is enabled (as it appears to be by default now)? I get a segfault in the same place every time when compiling a DIAGNOSTIC kernel when I leave it enabled. Just curious if this is just me or not... Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message Please post an error message and a traceback from GCC if possible. I might have the patch for that. -- Alexander Kabaev Ok, I think this is the correct backtrace: #0 0x0806ccee in cpp_ideq () #1 0x0806d31e in _cpp_lex_direct () #2 0x0806cfa5 in _cpp_lex_token () #3 0x080589e1 in cpp_macro_definition () #4 0x08058a69 in cpp_macro_definition () #5 0x08058ca7 in _cpp_handle_directive () #6 0x0806cfd4 in _cpp_lex_token () #7 0x08057dae in cpp_get_token () #8 0x08057ed9 in cpp_scan_nooutput () #9 0x080483e1 in do_preprocessing () #10 0x08048219 in main () #11 0x08048131 in _start () I'm seeing the segfault in the kernel make depend step, just as someone else reported. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
-mcpu=pentiumpro still evil?
Is anyone else still seeing Sig 11's from GCC when mcpu=pentiumpro is enabled (as it appears to be by default now)? I get a segfault in the same place every time when compiling a DIAGNOSTIC kernel when I leave it enabled. Just curious if this is just me or not... Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
fxp watchdog timeout patch
Sorry for the delay, here's the patch which should properly implement
watchdog timeout handling in the fxp driver. If you're one of the people
seeing the false watchdog timeout messages, please give this a whirl and
tell me how it worked.
Thanks,
Mike Silby Silbersack
--- if_fxp.cThu Jul 11 23:47:39 2002
+++ /home/silby/if_fxp.cFri Jul 19 00:36:57 2002
-1251,14 +1251,14
txp-mb_head = NULL;
}
sc-tx_queued--;
+ ifp-if_timer = 5;
}
sc-cbl_first = txp;
if (sc-tx_queued == 0) {
ifp-if_timer = 0;
if (sc-need_mcsetup)
fxp_mc_setup(sc);
- } else
- ifp-if_timer = 5;
+ }
/*
* Try to start more packets transmitting.
-1401,7 +1401,10
txp-mb_head = NULL;
}
sc-tx_queued--;
+ ifp-if_timer = 5;
}
+ if (sc-tx_queued == 0)
+ ifp-if_timer = 0;
sc-cbl_first = txp;
/*
* If we haven't received any packets in FXP_MAC_RX_IDLE seconds,
Re: zero copy code checkin in 2 days, new snapshot
On Mon, 24 Jun 2002, Kenneth D. Merry wrote: On Mon, Jun 24, 2002 at 01:17:03 -0500, Mike Silbersack wrote: On Sun, 23 Jun 2002, Kenneth D. Merry wrote: I'm planning on checking in the zero copy sockets code Tuesday evening, MDT. If there are any concerns, I'm more than willing to delay it. Out of curiousity, what happens when the page being write()n is a mmap'd page shared by multiple processes? Will the page be shared? That could be a big reduction in mbuf cluster usage on some http/ftp systems, I'd guess. The page would be shared, until one of the processes decides to write to it while it is still referenced in the kernel. If that happens, it'll get copied. Ken Cool, thttpd / others should benefit greatly then. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: zero copy code checkin in 2 days, new snapshot
On Mon, 24 Jun 2002, Andre Oppermann wrote: Mike Silbersack wrote: Cool, thttpd / others should benefit greatly then. The last time I checked thttpd didn't even use sendfile(2). It does use accf_http(9). Maybe kqueue(2) could speed it up further. -- Andre I thought that thttpd used kqueue (as of recent versions), and write()s from mmap'd files. I could be wrong, of course. (The program seems to evolve relatively quickly.) Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: zero copy code checkin in 2 days, new snapshot
On Sun, 23 Jun 2002, Kenneth D. Merry wrote: I'm planning on checking in the zero copy sockets code Tuesday evening, MDT. If there are any concerns, I'm more than willing to delay it. Out of curiousity, what happens when the page being write()n is a mmap'd page shared by multiple processes? Will the page be shared? That could be a big reduction in mbuf cluster usage on some http/ftp systems, I'd guess. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
-Current is stable enough for use again
FWIW, now that Peter has temporarily backed out his pmap-related changes, -current has stabilized again. Those who were having trouble with panics on boot (or within a few minutes after) with kernels built during the last day or two should definitely cvsup. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Today's panic on boot problem
I'm experiencing the same double panic on boot that PHK is now; are we the only ones, or is it just that nobody else has updated recently? Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Today's panic on boot problem
On Tue, 26 Feb 2002, Peter Wemm wrote: Mike Silbersack wrote: I'm experiencing the same double panic on boot that PHK is now; are we the only ones, or is it just that nobody else has updated recently? If you are not using acpica, then you're probably using vm86 for pcibios calls. I've been told that I've broken bios.c.. You may like to try reverting this change: peter 2002/02/25 13:42:23 PST Modified files: sys/i386/i386bios.c sys/i386/include pmap.h Log: Tidy up some warnings Revision ChangesPath 1.47 +7 -6 src/sys/i386/i386/bios.c 1.74 +1 -1 src/sys/i386/include/pmap.h Cheers, -Peter I reverted that change, and the double panic still occured. :| FWIW, you're correct in that I'm not using the acpi module. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Today's panic on boot problem
On Tue, 26 Feb 2002, Mike Silbersack wrote: I reverted that change, and the double panic still occured. :| FWIW, you're correct in that I'm not using the acpi module. Mike Silby Silbersack Using ACPI doesn't help here either. Hmph. Can I get a kernel dump that early in the boot process? The dumpon manpage doesn't suggest a way as far as I can tell. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Today's panic on boot problem
On Tue, 26 Feb 2002, David Wolfskill wrote: Date: Tue, 26 Feb 2002 19:46:59 + (GMT) From: Mike Silbersack [EMAIL PROTECTED] Using ACPI doesn't help here either. Hmph. Can I get a kernel dump that early in the boot process? The dumpon manpage doesn't suggest a way as far as I can tell. I was unable to get a dump, even though the panic dropped me into the debugger. Also, I only had the panic on my laptop -- the (SMP) build machine ran fine. Cheers, david (links to my resume at http://www.catwhisker.org/~david) Hm, sounds like UP got optimized out. Ok, I guess we wait... Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Today's panic on boot problem
On Tue, 26 Feb 2002, Peter Wemm wrote: Mike Silbersack wrote: Hm, sounds like UP got optimized out. Gah! That would be a first. :( Well, until I can build a working kernel, I'll just assume that it's a feature. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Today's panic on boot problem
On Tue, 26 Feb 2002, Peter Wemm wrote: FWIW, turning off PG_G see_ms to help. Change in pmap.c: #if !defined(SMP) || defined(ENABLE_PG_G) to: #if /*!defined(SMP) ||*/ defined(ENABLE_PG_G) and see how you go. This got me past atkbd0, but it is a very worrying sign. I now get a vnode related panic. :-( I've reread the changes about 50 times now and cannot for the life of me see why it works for SMP but not UP. I'm about ready to back it all out. Cheers, -Peter Disabling PG_G allows it to work here again as well. Given the problems we're experiencing, backing out the pmap changes of the last two days seems like a good idea. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Today's panic on boot problem
On Wed, 27 Feb 2002, Mike Silbersack wrote: Disabling PG_G allows it to work here again as well. Given the problems we're experiencing, backing out the pmap changes of the last two days seems like a good idea. Mike Silby Silbersack Well, I sorta take that back. The box has been up for ~5 minutes now, and the buildworld I started hasn't paniced it yet. I guess this is workable. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Newbie ddb question
I've been poking around in ddb in an attempt to work on some forkbomb/low memory problems, and I've found it extremely useful. There's one thing I can't figure out how to do that would be useful, though. Say that I have a process of interest tsleeping. Is there some way for me to get a backtrace of that process at the time it entered tsleep? In the case I'm creating, many processes are tsleeping on vmwait. As there are multiple places where such a wait can occur, I'd really like to see which codepaths are causing processes to enter that state. The trace command doesn't seem to have the necessary functionality to do this, so I was thinking that I might be able to simulate such an effect by putting a null pointer reference right before such calls, thereby creating a panic. Is this the best way to go, or is there some easier way to accomplish a similar effect? Thanks, Mike Silby' Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Newbie ddb question
On Thu, 14 Feb 2002, Bruce Evans wrote: On Thu, 14 Feb 2002, Mike Silbersack wrote: I've been poking around in ddb in an attempt to work on some forkbomb/low memory problems, and I've found it extremely useful. There's one thing I can't figure out how to do that would be useful, though. Say that I have a process of interest tsleeping. Is there some way for me to get a backtrace of that process at the time it entered tsleep? In the case I'm Try t pid. Bruce Ah, I see now. That never got MFC'd to -stable. I'll see if it's easily MFCable, or just move my testing over to -current. Thanks, Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: vnode_if.pl broken
On Fri, 4 Jan 2002, Bruce Evans wrote: Kernel builds now fail with the following error message: Global symbol $FreeBSD requires explicit package name at ./@/kern/vnode_if.pl line 82. Use of $* is deprecated at ./@/kern/vnode_if.pl line 82. Global symbol $FreeBSD requires explicit package name at ./@/kern/vnode_if.pl line 98. Execution of ./@/kern/vnode_if.pl aborted due to compilation errors. Bruce Crap. Ok, working on it. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: devfs question
On Sat, 27 Oct 2001, Poul-Henning Kamp wrote: Oops, error on my part; /proc does need to exist. So, I guess the question is this: Can devfs's error handling in the case of /dev being non-existant be improved? Barely, because without /dev, how do you plan to open the console ? Mkdir(/dev) isn't an option because the rootfs is mounted R/O. About the only thing one could do is make init exit(bla) and get a diagnostic from the kernel. It may make sense to do that. An error message would be sufficient; my concern was that someone might run into this and spend hours trying to figure out which of X variables was the problem. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: devfs question
On Sat, 27 Oct 2001, Poul-Henning Kamp wrote: An error message would be sufficient; my concern was that someone might run into this and spend hours trying to figure out which of X variables was the problem. Right, but the only way to get an error message is to let /sbin/init die and have the kernel print the message. /sbin/init cannot print the message when there is no /dev/console can it ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 I'm in over my head on this one; I really don't know anything about the boot process. Having init die if there's no /dev sounds acceptable to me. After all, if there's no /dev, the system looks like it's in trouble. (If this does not make sense, reference above paragraph.) Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Some interrupt coalescing tests
On Thu, 18 Oct 2001, Terry Lambert wrote: In the non-LRP case, the percentage drop in interrupt overhead is ~10% (as has been observed by others). THis makes sense, too, if you consider that NETISR driving of receives means less time in interrupt processing. If we multiply the 15% (100% - 85% = 15% in transmit) by 3 (12000/(12000-8000) = 100% / 33% = 3), then we get 45% in transmit in the non-LRP case. Hrm, so the reduction I saw is repeatable; good. If there are no objections, I'll fix up the whitespace later this week, fix the warning, and commit it. (The whitespace applies to the unified diff I created; your context one may have correct whitespace.) It would be nice if someone could confirm that slightly less than 1/2 of the looping is on the transmit side for a non-LRP kernel, but that's about what we should expect... If it is, I wonder if we could put a larger number of packets in the queue and disable transmit interrupts for a while. MMmm, dynamic queues. Sounds like something that would take a lot of work to get right, unfortunately. (Presumably, this tactic could be applied to most network cards, although all the better ones probably have really good transmit interrupt mitigation.) I'm really surprised abuse of the HTTP protocol itself in denial of service attacks isn't more common. Well, the attack your proposed would require symmetric bandwidth (if I understood it correctly), and is of course traceable. My guess would be that even script kiddies are smart enough to avoid attacks which could easily be traceable back to their drones. Even ignoring this, there's a pretty clear off the shelf hardware path to a full 10 gigabits, with PCI-X (8 gigabits times 2 busses gets you there, which is 25 times the largest UUNet hosting center pipe size today). Are you sure about that? I recently heard that Internet2 will be moving to 2.4Gbps backbones in the near future, and I assume that qwest wouldn't be willing to donate that bandwidth unless they had similar capabilities already. (They being generalized to all backbone providers.) Fair share is more a problem for slower interfaces without hardware coelescing, and software is an OK band-aid for them (IMO). I suspect that you will want to spend most of your CPU time doing processing, rather than interrupt handing, in any case. -- Terry Yep, probably. Are you implementing fair sharing soon? Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Some interrupt coalescing tests
On Sun, 14 Oct 2001, Terry Lambert wrote: The one thing I _would_ add -- though I'm waiting for it to be a problem before doing it -- is to limit the total number of packets processed per interrupt by keeping a running count. You would have to be _AMAZINGLY_ loaded to hit this, though; since it would mean absolutely continuous DMAs. I think it is self-limiting, should that happen, since once you are out of mbufs, you're out. The correct thing to do is probably to let it run out, but keep a seperate transmit reserve, so that you can process requests to completion. What probably should be done, if you have time, is to add a bit of profiling to your patch to find out how it helps most. I'm curious how many times it ends up looping, and also why it is looping (whether this is due to receive or transmit.) I think knowing this information would help optimize the drivers further, and perhaps suggest a tact we haven't thought of. I don't know if anyone has tested what happens to apache in a denial of service attack consisting of a huge number of partial GET requests that are incomplete, and so leave state hanging around in the HTTP server... I'm sure it would keel over and die, since it needs a process per socket. If you're talking about sockets in TIME_WAIT or such, see netkill.pl. Yes. Floyd and Druschel recommend using high and low watermarks on the amount of data pending processing in user space. The most common approach is to use a fair share scheduling algorithm, which reserves a certain amount of CPU for user space processing, but this is somewhat wasteful, if there is no work, since it denies quantum to the interrupt processing, potentially wrongly. I'm not sure such an algorithm would be wasteful - there must be data coming in to trigger such a huge amount of interrupts. I guess this would depend on how efficient your application is, how you set the limits, etc. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Some interrupt coalescing tests
On Sat, 13 Oct 2001, Terry Lambert wrote: Mike Silbersack wrote: One issue to be careful of here is that the removal of the tcptmpl actually causes a performance hit that wasn't there in the 4.3 code. My original complaint about tcptmpl taking up 256 instead of 60 bytes stands, but I'm more than half convinced that making it take up 60 bytes is OK... or at least is more OK than allocating and deallocating each time, and I don't yet have a better answer to the problem. 4.3 doesn't have this change, but 4.4 does. I need benchmarks to prove the slowdown, Terry. The testing I performed (which is limited, of course) showed no measureable speed difference. Remember that the only time the tcptempl mbuf ever gets allocated now is when a keepalive is sent, which is a rare event. The rest of the time, it's just copying the data from the preexisting structures over to the new packet. If you can show me that this method is slower, I will move it over to a zone allocated setup like you proposed. I'm not sure if the number was lower because the celeron couldn't run the flooder as quickly, or if the -current box was dropping packets. I suspect the latter, as the -current box was NOTICEABLY slowed down; I could watch systat refresh the screen. This is unfortunate; it's an effect that I expected with the -current code, because of the change to the interrupt processing path. To clarify here, the slowdown occurred both with and without the patch, right? The problem here is that when you hit livelock (full CPU utilization), then you are pretty much unable to do anything at all, unless the code path goes all the way to the top of the stack. Yep, the -current box livelocked with and without the patch. I'm not sure if -current is solely to blame, though. My -current box is using a PNIC, which incurs additional overhead relative to other tulip clones, according to the driver's comments. And the 3com in that box hasn't worked in a while... maybe I should try debugging that so I have an additional test point. The conclusion? I think that the dc driver does a good enough job of grabbing multiple packets at once, and won't be helped by Terry's patch except in a few very cases. 10% is a good improvement; my gut feeling is that it would have been less than that. This is actually good news for me, since it means that my 30% number is bounded by the user space program not being run (in other words, I should be able to get considerably better performance, using a weighted fair share scheduler). As long as it doesn't damage performance, I think that it's proven itself. Hm, true, I guess the improvement is respectable. My thought is mostly that I'm not sure how much it's extending the performance range of a system; testing with more varied packet loads as suggested by Alfred would help tell us the answer to this. In fact, I have a sneaky suspicion that Terry's patch may increase bus traffic slightly. I'm not sure how much of an issue this is, perhaps Bill or Luigi could comment. This would be interesting to me, as well. I gave Luigi an early copy of the patch to play with a while ago, and also copied Bill. I'm interested in how you think it could increase traffic; the only credible reason I've been able to come up with is the ability to push more packets through, when they would otherwise end up being dropped because of the queue full condition -- if this is the case, the bus traffic is real work, and not additonal overhead. The extra polling of the bus in cases where there are no additional packets to grab is what I was wondering about. I guess in comparison to the quantity of packet data going by, it's not a real issue. In short, if we're going to try to tackle high interrupt load, it should be done by disabling interrupts and going to polling under high load; I would agree with this, except that it's only really a useful observation if FreeBSD is being used as purely a network processor. Without interrupts, the polling will take a significant portion of the available CPU to do, and you can't burn that CPU if, for example, you have an SSL card that does your handshakes, but you need to run the SSL sessions themselves up in user space. Straight polling isn't necessarily the solution I was thinking of, but rather some form of interrupt disabling at high rates. For example, if the driver were to keep track of how many interrupts/second it was taking, perhaps it could up the number of receive buffers from 64 to something higher, then disable the card's interrupt and set a callback to run in a short bit of time at which point interrupts would be reenabled and the interrupt handler would be run. Ideally, this could reduce the number of interrupts greatly, increasing efficiency under load. Paired with this could be receive polling during transmit, something which does not seem to be done at current, if I'm reading correctly. I'm not sure
Some interrupt coalescing tests
Well, I've been watching everyone argue about the value of interrupt
coalescing in the net drivers, so I decided to port terry's patch to 4.4
-current to see what the results are. The patch included applies cleanly
to 4.4's if_dc, and will apply to -current with a one line change.
Whitespace is horrible, I copied and pasted the original patch, used patch
-l, etc.
The test setup I used was as follows:
Duron 600, PNIC, running -current
Celeron 450, ADMtek tulip-clone, running -stable
The network is 100mbps, switched. To simulate load, I used a syn flooder
aimed at an unused port. icmp/rst response limiting was enabled.
With the -current box attacking the -stable box, I was able to notice a
slight drop in interrupts/second with the patch applied. The number of
packets was ~57000/second.
Before: ~46000 ints/sec, 57-63% processor usage due to interrupts.
After: ~38000 ints/sec, 50-60% processor usage due to interrupts.
In both cases, the box felt responsive.
With the -stable box attacking the -current box, the patch made no
difference. The box bogged down at only ~25000 ints/sec, and response
limiting reported the number of packets to be ~44000/second.
I'm not sure if the number was lower because the celeron couldn't run the
flooder as quickly, or if the -current box was dropping packets. I
suspect the latter, as the -current box was NOTICEABLY slowed down; I
could watch systat refresh the screen.
The conclusion? I think that the dc driver does a good enough job of
grabbing multiple packets at once, and won't be helped by Terry's patch
except in a few very cases. In fact, I have a sneaky suspicion that
Terry's patch may increase bus traffic slightly. I'm not sure how much of
an issue this is, perhaps Bill or Luigi could comment.
In short, if we're going to try to tackle high interrupt load, it should
be done by disabling interrupts and going to polling under high load;
the patch proposed here isn't worth the extra complexity.
I suppose this would all change if we were using LRP and doing lots of
processing in the interrupt handler... but we aren't.
Mike Silby Silbersack
--- if_dc.c.origThu Oct 11 01:39:05 2001
+++ if_dc.c Thu Oct 11 01:39:30 2001
@@ -193,8 +193,8 @@
static int dc_coal __P((struct dc_softc *, struct mbuf **));
static void dc_pnic_rx_bug_war __P((struct dc_softc *, int));
static int dc_rx_resync__P((struct dc_softc *));
-static void dc_rxeof __P((struct dc_softc *));
-static void dc_txeof __P((struct dc_softc *));
+static int dc_rxeof __P((struct dc_softc *));
+static int dc_txeof __P((struct dc_softc *));
static void dc_tick__P((void *));
static void dc_tx_underrun __P((struct dc_softc *));
static void dc_intr__P((void *));
@@ -2302,7 +2302,7 @@
* A frame has been uploaded: pass the resulting mbuf chain up to
* the higher level protocols.
*/
-static void dc_rxeof(sc)
+static int dc_rxeof(sc)
struct dc_softc *sc;
{
struct ether_header*eh;
@@ -2311,6 +2311,7 @@
struct dc_desc *cur_rx;
int i, total_len = 0;
u_int32_t rxstat;
+ int cnt = 0;
ifp = sc-arpcom.ac_if;
i = sc-dc_cdata.dc_rx_prod;
@@ -2355,7 +2356,7 @@
continue;
} else {
dc_init(sc);
- return;
+ return(cnt);
}
}
@@ -2379,6 +2380,7 @@
/* Remove header from mbuf and pass it on. */
m_adj(m, sizeof(struct ether_header));
ether_input(ifp, eh, m);
+ cnt++;
}
sc-dc_cdata.dc_rx_prod = i;
@@ -2389,12 +2391,13 @@
* the list buffers.
*/
-static void dc_txeof(sc)
+static int dc_txeof(sc)
struct dc_softc *sc;
{
struct dc_desc *cur_tx = NULL;
struct ifnet*ifp;
int idx;
+ int cnt = 0;
ifp = sc-arpcom.ac_if;
@@ -2452,7 +2455,7 @@
ifp-if_collisions++;
if (!(txstat DC_TXSTAT_UNDERRUN)) {
dc_init(sc);
- return;
+ return(cnt);
}
}
@@ -2466,13 +2469,14 @@
sc-dc_cdata.dc_tx_cnt--;
DC_INC(idx, DC_TX_LIST_CNT);
+ cnt++;
}
sc-dc_cdata.dc_tx_cons = idx;
if (cur_tx != NULL)
ifp-if_flags = ~IFF_OACTIVE;
- return;
+ return(cnt);
}
static void dc_tick(xsc)
@@ -2612,6 +2616,7 @@
struct dc_softc *sc;
Re: Some interrupt coalescing tests
On Fri, 12 Oct 2001, Alfred Perlstein wrote: The network is 100mbps, switched. To simulate load, I used a syn flooder aimed at an unused port. icmp/rst response limiting was enabled. Actually, you might want to leave that on, it will generate more load. I considered leaving it on, but I'm not sure if that would be constructive or not. The primary problem with doing that is related to my test setup - as we see from the stable - current attack, my current box couldn't take the interrupt load of that many incoming packets, which would slow down the outgoing packets. If I had a better test setup, I'd like to try that. Before: ~46000 ints/sec, 57-63% processor usage due to interrupts. After: ~38000 ints/sec, 50-60% processor usage due to interrupts. In both cases, the box felt responsive. You need to get real hardware to run these tests, obviously you aren't saturating your line. I would suspect a better test would be to see how many pps you get can at the point where cpu utlization reaches 100%. Basically start at a base of 60,000pps, and see how many more it takes to drive them both to 100%. Even your limited tests show a mean improvement of something like 10%. 10% isn't earth shattering, but it is a signifigant improvement. Yes, there is some improvement, but I'm not sure that the actual effect is worthwhile. Even with the 10% decrease, you're still going to kill the box if the interrupt count goes much higher. If you can setup a 4.4+this patch test of some sort with varying loads to see the effect, maybe we could characterize the effect of the patch more. With my setup, I don't think I can really take this testing any further. Mike Silby Silbersack To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Patch to enable asm cores w/openssl
http://www.silby.com/patches/openssl-asm.patch I've just thrown together a patch that enables the asm cores (where available) in openssl. I've tested it on a p5, k6, and p6 - all show good improvements (from 1.1x - 2.0x, depending on the core.) SHA1, blowfish, and cast aren't running as fast as they could - doing so would make the resulting lib not work on 386es. Nonetheless, there's still a marked improvement over the C versions of the code. There are two things I can't easily test, which I was hoping I could get some help on before I bug kris about committing this. 1. Operation on 386es. I don't have one of these around, but if someone has one running 4.x or 5.x and could test to make sure that the updated libcrypto works properly on it, I'd feel much safer. 2. Speed on Athlons. I don't have one of these handy, and I'm curious if the asm cores cause any regressions in speed. For those not familiar with openssl, the process of benchmarking is simple - just type "openssl speed". Any comments about how to improve my make syntax would also be helpful, if one is so inclined. Thanks, Mike "Silby" Silbersack (Please CC me on followups, I'm not on the list.) To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
