Crash in dummynet.
Hello list, I'm observing recurring crashes with dummynet on 8.1-RELEASE. Abnormal things being done on the system are: /sbin/ipfw pipe list pipestats-`date +%Y%m%d-%H%M%S` being done every minute. There are over 2000 pipes in the system. Anyway, to the point: is there any resource I could read as to how to prepare the system to catch correct debugging info at next crash? What other helpful information may I provide? Thanks. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Crash in dummynet.
On Tue, Aug 17, 2010 at 01:08:43PM +0200, Pawel Tyll wrote: Hello list, I'm observing recurring crashes with dummynet on 8.1-RELEASE. Abnormal things being done on the system are: /sbin/ipfw pipe list pipestats-`date +%Y%m%d-%H%M%S` being done every minute. There are over 2000 pipes in the system. Anyway, to the point: is there any resource I could read as to how to prepare the system to catch correct debugging info at next crash? What other helpful information may I provide? a backtrace and the detailed version of the ipfw+dummynet files will help cheers luigi ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: freebsd-stable Digest, Vol 370, Issue 2
Well in normal use user applications are the ones that put the load on the CPU. The kernel is pretty much lightweight in size and work when non kernel intensive tasks are at hand (like gigabit links in the networks, etc). The good news is that the kernel will have its own core to play with. On 08/17/2010 08:00 AM, freebsd-stable-requ...@freebsd.org wrote: Send freebsd-stable mailing list submissions to freebsd-stable@freebsd.org To subscribe or unsubscribe via the World Wide Web, visit http://lists.freebsd.org/mailman/listinfo/freebsd-stable or, via email, send a message with subject or body 'help' to freebsd-stable-requ...@freebsd.org You can reach the person managing the list at freebsd-stable-ow...@freebsd.org When replying, please edit your Subject line so it is more specific than Re: Contents of freebsd-stable digest... Today's Topics: 1. Re: Inconsistent IO performance (Ivan Voras) 2. Re: Inconsistent IO performance (Kevin Oberman) 3. STABLE kernel panic: privileged instruction fault (Alexey Tarasov) 4. Re: Inconsistent IO performance (Jeremy Chadwick) 5. Re: Inconsistent IO performance (Jeremy Chadwick) 6. Re: STABLE kernel panic: privileged instruction fault (Kostik Belousov) 7. Re: STABLE kernel panic: privileged instruction fault (Alexey Tarasov) 8. Re: STABLE kernel panic: privileged instruction fault (Kostik Belousov) 9. Re: STABLE kernel panic: privileged instruction fault (Alexey Tarasov) 10. Re: STABLE kernel panic: privileged instruction fault (Kostik Belousov) 11. Re: STABLE kernel panic: privileged instruction fault (Alexey Tarasov) 12. Re: RELENG_7 em problems (and RELENG_8) (Mike Tancsa) 13. Performance AMD Phenom II X6 1090T (Vladislav V. Prodan) 14. Crash in dummynet. (Pawel Tyll) 15. Re: Crash in dummynet. (Luigi Rizzo) -- Message: 1 Date: Mon, 16 Aug 2010 15:03:23 +0200 From: Ivan Vorasivo...@freebsd.org Subject: Re: Inconsistent IO performance To:freebsd-stable@freebsd.org Message-ID:i4bcuv$21...@dough.gmane.org Content-Type: text/plain; charset=UTF-8 On 13.8.2010 18:01, Kevin Oberman wrote: For some time I have seen very odd issues with IO performance on 8-Stable. Going back to November of last year when 8.0 was released, I see variations of up to 22% in identical operations. This is not a degradation as the performance moves up and down. In 8.0-8.1 span of time there was some work on the ata driver to make it use MAXPHYS (128 KiB) transfer sizes instead of 64 KiB. Modifying this will involve changing and recompiling the kernel but if you want to try something and the hardware is SATA you might try the new AHCI driver (ada). http://ivoras.net/blog/tree/2009-11-17.trying-ahci-in-8.0.html -- Message: 2 Date: Mon, 16 Aug 2010 07:46:38 -0700 From: Kevin Obermanober...@es.net Subject: Re: Inconsistent IO performance To: Ivan Vorasivo...@freebsd.org Cc:freebsd-stable@freebsd.org Message-ID:20100816144638.4acf81c...@ptavv.es.net From: Ivan Vorasivo...@freebsd.org Date: Mon, 16 Aug 2010 15:03:23 +0200 Sender:owner-freebsd-sta...@freebsd.org On 13.8.2010 18:01, Kevin Oberman wrote: For some time I have seen very odd issues with IO performance on 8-Stable. Going back to November of last year when 8.0 was released, I see variations of up to 22% in identical operations. This is not a degradation as the performance moves up and down. In 8.0-8.1 span of time there was some work on the ata driver to make it use MAXPHYS (128 KiB) transfer sizes instead of 64 KiB. Modifying this will involve changing and recompiling the kernel but if you want to try something and the hardware is SATA you might try the new AHCI driver (ada). http://ivoras.net/blog/tree/2009-11-17.trying-ahci-in-8.0.html Thanks. I appreciate the suggestion. I am running a 8-Stable kernel from August 9, so I think I should be OK on this. IS there a requirement to set some parameter in the kernel config to take advantage of this? While the ThinkPad has a SATA ICH6-M chip-set, it does not provide or any SATA connections. Both SATA ports a run to a SATA/PATA converter chip and the only 2 physical connections available are PATA. I am assuming that this is because 2.5 in. SATA drives were pretty much unavailable when this system was shipped. This was the last of the T43 series and was dropped from the product line by Lenovo about a month after I got it, to be replaced by T60 systems running Core2 chips and using SATA drives. Just lousy timing almost 4 years ago. Thanks again! ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: svn commit: r209611 - head/sys/dev/e1000
On 1 July 2010 02:13, Jack Vogel jfvo...@gmail.com wrote: On Wed, Jun 30, 2010 at 2:50 PM, Julian Elischer jul...@elischer.orgwrote: On 6/30/10 10:26 AM, Jack F Vogel wrote: Author: jfv Date: Wed Jun 30 17:26:47 2010 New Revision: 209611 URL: http://svn.freebsd.org/changeset/base/209611 Log: SR-IOV support added to igb What this provides is support for the 'virtual function' interface that a FreeBSD VM may be assigned from a host like KVM on Linux, or newer versions of Xen with such support. When the guest is set up with the capability, a special limited function 82576 PCI device is present in its virtual PCI space, so with this driver installed in the guest that device will be detected and function nearly like the bare metal, as it were. The interface is only allowed a single queue in this configuration however initial performance tests have looked very good. Enjoy!! do these extra devices turn up in a standard ifconfig output? in other words, can we assign them to jails using vimage? They only show up if configured in the PF host, for instance if using Linux and KVM (I did develop and test with Fedora 13) you must load the igb driver there specifying that you want vf's created and how many. Next in the management of the guest you need to assign one of these vf devices to the guest. After you do all that and load this igb driver then yes, it will look just like a standard igbX device. Hi, Jack. I set up qemu-kvm on openSUSE 11.3 with 82576 PCI device as you described. Guest fails to attach with: igb0: Intel(R) PRO/1000 Network Connection version - 2.0.1 mem 0xf206-0xf2063fff,0xf2064000-0xf2067fff at device 5.0 on pci0 igb0: Unable to allocate bus resource: interrupt device_attach: igb0 attach returned 6 i...@pci0:0:5:0:class=0x02 card=0xa03c8086 chip=0x10ca8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet cap 11[40] = MSI-X supports 3 messages in map 0x1c Did I missed something? -- wbr, pluknet ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: svn commit: r209611 - head/sys/dev/e1000
Cool the first person to actually try and use it :) Yes, there's one key thing you have to do right now that's not documented, because of the simplistic PCI structure the guest has the kernel blacklists it from using MSIX. SO, what you need to do is set the honor_blacklist (that's not the complete string, use sysctl -a |grep blacklist to find it) and set that to 0. It needs to be set at boot. That should get you running. Jack On Tue, Aug 17, 2010 at 8:18 AM, pluknet pluk...@gmail.com wrote: On 1 July 2010 02:13, Jack Vogel jfvo...@gmail.com wrote: On Wed, Jun 30, 2010 at 2:50 PM, Julian Elischer jul...@elischer.org wrote: On 6/30/10 10:26 AM, Jack F Vogel wrote: Author: jfv Date: Wed Jun 30 17:26:47 2010 New Revision: 209611 URL: http://svn.freebsd.org/changeset/base/209611 Log: SR-IOV support added to igb What this provides is support for the 'virtual function' interface that a FreeBSD VM may be assigned from a host like KVM on Linux, or newer versions of Xen with such support. When the guest is set up with the capability, a special limited function 82576 PCI device is present in its virtual PCI space, so with this driver installed in the guest that device will be detected and function nearly like the bare metal, as it were. The interface is only allowed a single queue in this configuration however initial performance tests have looked very good. Enjoy!! do these extra devices turn up in a standard ifconfig output? in other words, can we assign them to jails using vimage? They only show up if configured in the PF host, for instance if using Linux and KVM (I did develop and test with Fedora 13) you must load the igb driver there specifying that you want vf's created and how many. Next in the management of the guest you need to assign one of these vf devices to the guest. After you do all that and load this igb driver then yes, it will look just like a standard igbX device. Hi, Jack. I set up qemu-kvm on openSUSE 11.3 with 82576 PCI device as you described. Guest fails to attach with: igb0: Intel(R) PRO/1000 Network Connection version - 2.0.1 mem 0xf206-0xf2063fff,0xf2064000-0xf2067fff at device 5.0 on pci0 igb0: Unable to allocate bus resource: interrupt device_attach: igb0 attach returned 6 i...@pci0:0:5:0:class=0x02 card=0xa03c8086 chip=0x10ca8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet cap 11[40] = MSI-X supports 3 messages in map 0x1c Did I missed something? -- wbr, pluknet ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: NFS stalling on 8.1-STABLE
On Sun, 15 Aug 2010 17:11:01 -0400 (EDT) Rick Macklem rmack...@uoguelph.ca wrote: Hi all, I have five front end web servers that all mount their content from the same server via NFS. If I stress the link on any one of the machines (eg: copy a large directory with a lot of files to/from the mounted file system) the client will pause. That is, all processes trying to access that mount will freeze. The log files with hundreds or thousands of nfs server not responding / is alive again messages. After 60 seconds it returns to normal, unless the load is still there in which case it continues to pause. The 60sec delay suggests that the client is doing a TCP reconnect. I'd suggest that you look at a packet trace in wireshark (it knows how to decode NFS packets) and see if there are new TCP connections (SYN, SYN-ACK,...) being made. If that is what is happening, I suspect it is NIC driver related, but it is really hard to say. I'll try this if/when it happens again. If you can try a network interface of a different type (not em) that will check to see if it is an em(4) issue. Unfortunately I don't have any non-em cards around. Alternately, you could try turning off the TSO and checksum offload stuff for the em(4) and see if that helps. Hmm, interesting. The four machines that seem to be working (so far) have these enabled by default. The fifth one has checksums enabled, but not TSO. Doesn't appear to support it. I also tried switching from TCP to UDP. This seems to be working (so far) on four of the clients (which happen to be identical load balanced machines), but on the fifth one (which serves a different purpose) I'm getting something really weird. Instead of locking up periodically as before, it's actually losing the mount. For example, a 'df' doesn't include the mounted system. If I try to access the mounted system (with 'ls' for example) I get an Input / output error message. I can remount it, but only after I force a dismount. Mark ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: NFS stalling on 8.1-STABLE
On Sun, 15 Aug 2010 23:35:50 -0700 Jeremy Chadwick free...@jdc.parodius.com wrote: On Thu, Aug 12, 2010 at 10:35:49AM -0700, Mark Morley wrote: I have five front end web servers that all mount their content from the same server via NFS. If I stress the link on any one of the machines (eg: copy a large directory with a lot of files to/from the mounted file system) the client will pause. That is, all processes trying to access that mount will freeze. The log files with hundreds or thousands of nfs server not responding / is alive again messages. After 60 seconds it returns to normal, unless the load is still there in which case it continues to pause. This has only started happening since I upgraded the client machines to 8.1-STABLE (previously four of them were 8.0 and one was 7.3). The server is 7.1-RELEASE-p11. No other changes have taken place in terms of hardware or software or mount options, etc. All nics involved are gigabit em cards, and they are on a private network (web access to the boxes is via an external interface). Are there any indications in dmesg that the NIC is responsible, e.g. interface down/up, etc.? No, nothing like that. Does switching to UDP-based NFS solve the problem for you? Trying that now for the past 24 hours or so. Four of the machine seem ok so far, but the fifth one has started dropping the mount entirely. Access to it gives an Input / output error message. Forcing a dismount and remounting brings it back. What OS version (uname -a) and NIC are used on the NFS server? FreeBSD xxx 7.1-RELEASE-p11 FreeBSD 7.1-RELEASE-p11 #0: Wed May 26 03:20:59 PDT 2010 r...@xxx:/usr/obj/usr/src/sys/CUSTOM i386 NICs are em Can you please provide the following output from one of the client machines running 8.1-STABLE with gigE em(4)? You can X-out machine names, MAC addresses, and IP addresses/netblocks if need be. * uname -a FreeBSD xxx 8.1-STABLE FreeBSD 8.1-STABLE #0: Tue Jul 27 16:27:44 PDT 2010 r...@xxx:/usr/obj/usr/src/sys/CUSTOM amd64 * ifconfig emX (where X is the interface number which would be used for NFS) em0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=209bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC ether 00:0e:0c:85:d5:0d inet 192.168.1.30 netmask 0xff00 broadcast 192.168.1.255 media: Ethernet 1000baseT full-duplex status: active * netstat -idn -I emX NameMtu Network Address Ipkts Ierrs IdropOpkts Oerrs Coll Drop em01500 Link#1 00:0e:0c:85:d5:0d 39913814 2 0 39949943 0 00 em01500 192.168.1.0/2 192.168.1.30 39944016 - - 39949664 - -- * pciconf -lvc (provide only the data for emX please) e...@pci0:1:6:0: class=0x02 card=0x13768086 chip=0x107c8086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' device = 'Gigabit Ethernet Controller (Copper) rev 5 (82541PI)' class = network subclass = ethernet cap 01[dc] = powerspec 2 supports D0 D3 current D0 cap 07[e4] = PCI-X supports 2048 burst read, 1 split transaction * vmstat -i interrupt total rate irq1: atkbd0 239 0 irq16: em0 36746591883 irq18: em1 12658607304 irq21: ohci0 2 0 irq22: ehci0 528002 12 irq23: atapci1 2334936 56 cpu0: timer 83207296 2000 cpu1: timer 83207289 2000 Total 218682962 5256 * sysctl hw.pci hw.pci.usb_early_takeover: 1 hw.pci.honor_msi_blacklist: 1 hw.pci.enable_msix: 1 hw.pci.enable_msi: 1 hw.pci.do_power_resume: 1 hw.pci.do_power_nodriver: 0 hw.pci.enable_io_modes: 1 hw.pci.default_vgapci_unit: -1 hw.pci.host_mem_start: 2147483648 hw.pci.mcfg: 1 * As root, run sysctl dev.em.X.stats=1 then do dmesg and provide the output for NIC statistics (will start with emX:) em0: Excessive collisions = 0 em0: Sequence errors = 0 em0: Defer count = 52 em0: Missed Packets = 0 em0: Receive No Buffers = 0 em0: Receive Length Errors = 0 em0: Receive errors = 1 em0: Crc errors = 1 em0: Alignment errors = 0 em0: Collision/Carrier extension errors = 0 em0: RX overruns = 0 em0: watchdog timeouts = 0 em0: RX MSIX IRQ = 0 TX MSIX IRQ = 0 LINK MSIX IRQ = 0 em0: XON Rcvd = 54 em0: XON Xmtd = 0 em0: XOFF Rcvd = 54 em0: XOFF Xmtd = 0 em0: Good Packets Rcvd = 39915088 em0: Good Packets Xmtd = 39951839 Mark ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: RELENG_7 em problems (and RELENG_8)
On Mon, Aug 16, 2010 at 05:07:11PM -0400, Mike Tancsa wrote: Hi Jack, FYI, I am still seeing this same problem on RELENG_8 (code as of today). Unfortunately I cant try Pyun's patch since the underlying code has changed since then. e...@pci0:3:0:0: class=0x02 card=0x34ec8086 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c pci3: ACPI PCI bus on pcib3 em4: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x1000-0x101f mem 0xb190-0xb191,0xb192-0xb1923fff irq 16 at device 0.0 on pci3 em4: Using MSI interrupt em4: [FILTER] em4: Ethernet address: 00:15:17:ed:3e:c4 Here is updated patch for HEAD and stable/8. http://people.freebsd.org/~yongari/em.csum_tso.20100817.patch It seems to work as expected under my limited environments. If you're using multiple Tx queues with em(4) it would be better to disable Tx checksum offloading as driver always have to create a new checksum context for each frame. This will effectively disable pipelined Tx data DMA which in turn greatly slows down Tx performance for small sized frames. The reason driver have to create a new checksum context when it uses multiple Tx queues comes from hardware limitation. The controller tracks only for the last context descriptor that was written such that driver does not know the state of checksum context configured in other Tx queue. Hope this helps. ---Mike At 03:36 PM 7/2/2010, Pyun YongHyeon wrote: On Fri, Jul 02, 2010 at 01:39:22PM -0400, Mike Tancsa wrote: Hi Jack, Just a followup to the email below. I now saw what appears to be the same problem on RELENG_8, but on a different nic and with VLANs. So not sure if this is a general em problem, a problem specific to some em NICs, or a TSO problem in general. The issue seemed to be triggered when I added a new vlan based on e...@pci0:14:0:0:class=0x02 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel PRO/1000 PL Network Adaptor (82573L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) pci14: ACPI PCI bus on pcib5 em3: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x6000-0x601f mem 0xe830-0xe831 irq 17 at device 0.0 on pci14 em3: Using MSI interrupt em3: [FILTER] em3: Ethernet address: 00:30:48:9f:eb:81 em3: flags=8943UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST metric 0 mtu 1500 options=2098VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC ether 00:30:48:9f:eb:81 inet 10.255.255.254 netmask 0xfffc broadcast 10.255.255.255 media: Ethernet autoselect (1000baseT full-duplex) status: active I had to disable tso, rxcsum and txsum in order to see the devices on the other side of the two vlans trunked off em3. Unfortunately, the other sides were switches 100km and 500km away so I didnt have any tcpdump capabilities to diagnose the issue. I had already created one vlan off this NIC and all was fine. A few weeks later, I added a new one and I could no longer telnet into the remote switches from the local machine But, I could telnet into the switches from machines not on the problem box. Hence, it would appear to be a general TSO issue no ? I disabled tso on the nic (I didnt disable net.inet.tcp.tso as I forgot about that).. Still nothing. I could always ping the remote devices, but no tcp services. I then remembered this issue from before, so I tried disabling tso on the NIC. Still nothing. Then I disabled rxcsum and txcsum and I could then telnet into the remote devices. This newly observed issue was from a buildworld on Mon Jun 14 11:29:12 EDT 2010. I will try and recreate the issue locally again to see if I can trigger the problem on demand. Any thoughts on what it might be ? Perhaps an issue specific to certain em nics ? http://www.freebsd.org/cgi/query-pr.cgi?pr=141843 I'm not sure whether you're seeing the same issue though. I didn't have chance to try latest em(4) on stable/7. Mike Tancsa, tel +1 519 651 3400 Sentex Communications,m...@sentex.net Providing Internet since 1994www.sentex.net Cambridge, Ontario Canada
Re: RELENG_7 em problems (and RELENG_8)
Hmmm, interesting, I'll have to have some testing done, maybe for the 574 it should automagically disable CSUM? Jack On Tue, Aug 17, 2010 at 11:52 AM, Pyun YongHyeon pyu...@gmail.com wrote: On Mon, Aug 16, 2010 at 05:07:11PM -0400, Mike Tancsa wrote: Hi Jack, FYI, I am still seeing this same problem on RELENG_8 (code as of today). Unfortunately I cant try Pyun's patch since the underlying code has changed since then. e...@pci0:3:0:0: class=0x02 card=0x34ec8086 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c pci3: ACPI PCI bus on pcib3 em4: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x1000-0x101f mem 0xb190-0xb191,0xb192-0xb1923fff irq 16 at device 0.0 on pci3 em4: Using MSI interrupt em4: [FILTER] em4: Ethernet address: 00:15:17:ed:3e:c4 Here is updated patch for HEAD and stable/8. http://people.freebsd.org/~yongari/em.csum_tso.20100817.patchhttp://people.freebsd.org/%7Eyongari/em.csum_tso.20100817.patch It seems to work as expected under my limited environments. If you're using multiple Tx queues with em(4) it would be better to disable Tx checksum offloading as driver always have to create a new checksum context for each frame. This will effectively disable pipelined Tx data DMA which in turn greatly slows down Tx performance for small sized frames. The reason driver have to create a new checksum context when it uses multiple Tx queues comes from hardware limitation. The controller tracks only for the last context descriptor that was written such that driver does not know the state of checksum context configured in other Tx queue. Hope this helps. ---Mike At 03:36 PM 7/2/2010, Pyun YongHyeon wrote: On Fri, Jul 02, 2010 at 01:39:22PM -0400, Mike Tancsa wrote: Hi Jack, Just a followup to the email below. I now saw what appears to be the same problem on RELENG_8, but on a different nic and with VLANs. So not sure if this is a general em problem, a problem specific to some em NICs, or a TSO problem in general. The issue seemed to be triggered when I added a new vlan based on e...@pci0:14:0:0:class=0x02 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel PRO/1000 PL Network Adaptor (82573L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) pci14: ACPI PCI bus on pcib5 em3: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x6000-0x601f mem 0xe830-0xe831 irq 17 at device 0.0 on pci14 em3: Using MSI interrupt em3: [FILTER] em3: Ethernet address: 00:30:48:9f:eb:81 em3: flags=8943UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST metric 0 mtu 1500 options=2098VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC ether 00:30:48:9f:eb:81 inet 10.255.255.254 netmask 0xfffc broadcast 10.255.255.255 media: Ethernet autoselect (1000baseT full-duplex) status: active I had to disable tso, rxcsum and txsum in order to see the devices on the other side of the two vlans trunked off em3. Unfortunately, the other sides were switches 100km and 500km away so I didnt have any tcpdump capabilities to diagnose the issue. I had already created one vlan off this NIC and all was fine. A few weeks later, I added a new one and I could no longer telnet into the remote switches from the local machine But, I could telnet into the switches from machines not on the problem box. Hence, it would appear to be a general TSO issue no ? I disabled tso on the nic (I didnt disable net.inet.tcp.tso as I forgot about that).. Still nothing. I could always ping the remote devices, but no tcp services. I then remembered this issue from before, so I tried disabling tso on the NIC. Still nothing. Then I disabled rxcsum and txcsum and I could then telnet into the remote devices. This newly observed issue was from a buildworld on Mon Jun 14 11:29:12 EDT 2010. I will try and recreate the issue locally again to see if I can trigger the problem on demand. Any thoughts on what it might be ? Perhaps an issue specific to certain em nics ? http://www.freebsd.org/cgi/query-pr.cgi?pr=141843 I'm not sure whether you're seeing the same issue though. I didn't have chance
Re: RELENG_7 em problems (and RELENG_8)
On Tue, Aug 17, 2010 at 11:52:08AM -0700, Pyun YongHyeon wrote: On Mon, Aug 16, 2010 at 05:07:11PM -0400, Mike Tancsa wrote: Hi Jack, FYI, I am still seeing this same problem on RELENG_8 (code as of today). Unfortunately I cant try Pyun's patch since the underlying code has changed since then. e...@pci0:3:0:0: class=0x02 card=0x34ec8086 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c pci3: ACPI PCI bus on pcib3 em4: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x1000-0x101f mem 0xb190-0xb191,0xb192-0xb1923fff irq 16 at device 0.0 on pci3 em4: Using MSI interrupt em4: [FILTER] em4: Ethernet address: 00:15:17:ed:3e:c4 Here is updated patch for HEAD and stable/8. http://people.freebsd.org/~yongari/em.csum_tso.20100817.patch It seems to work as expected under my limited environments. If you're using multiple Tx queues with em(4) it would be better to disable Tx checksum offloading as driver always have to create a new checksum context for each frame. This will effectively disable pipelined Tx data DMA which in turn greatly slows down Tx performance for small sized frames. The reason driver have to create a new checksum context when it uses multiple Tx queues comes from hardware limitation. The controller tracks only for the last context descriptor that was written such that driver does not know the state of checksum context configured in other Tx queue. Hope this helps. For igb(4) controllers, it seems we also need a way to avoid creating a new checksum context for every Tx frame as we did in em(4). Unlike em(4) controllers, igb(4) seems to maintain context per queue such that we can safely reuse previously configured context for a queue. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: RELENG_7 em problems (and RELENG_8)
I believe the requirement of a context descriptor for most frames in the igb driver is just the way the hardware works, I've looked over the Linux driver again and it looks like they require the same. I don't believe its a big deal, just the added descriptor for the frame. Jack On Tue, Aug 17, 2010 at 12:14 PM, Pyun YongHyeon pyu...@gmail.com wrote: On Tue, Aug 17, 2010 at 11:52:08AM -0700, Pyun YongHyeon wrote: On Mon, Aug 16, 2010 at 05:07:11PM -0400, Mike Tancsa wrote: Hi Jack, FYI, I am still seeing this same problem on RELENG_8 (code as of today). Unfortunately I cant try Pyun's patch since the underlying code has changed since then. e...@pci0:3:0:0: class=0x02 card=0x34ec8086 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c pci3: ACPI PCI bus on pcib3 em4: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x1000-0x101f mem 0xb190-0xb191,0xb192-0xb1923fff irq 16 at device 0.0 on pci3 em4: Using MSI interrupt em4: [FILTER] em4: Ethernet address: 00:15:17:ed:3e:c4 Here is updated patch for HEAD and stable/8. http://people.freebsd.org/~yongari/em.csum_tso.20100817.patchhttp://people.freebsd.org/%7Eyongari/em.csum_tso.20100817.patch It seems to work as expected under my limited environments. If you're using multiple Tx queues with em(4) it would be better to disable Tx checksum offloading as driver always have to create a new checksum context for each frame. This will effectively disable pipelined Tx data DMA which in turn greatly slows down Tx performance for small sized frames. The reason driver have to create a new checksum context when it uses multiple Tx queues comes from hardware limitation. The controller tracks only for the last context descriptor that was written such that driver does not know the state of checksum context configured in other Tx queue. Hope this helps. For igb(4) controllers, it seems we also need a way to avoid creating a new checksum context for every Tx frame as we did in em(4). Unlike em(4) controllers, igb(4) seems to maintain context per queue such that we can safely reuse previously configured context for a queue. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: RELENG_7 em problems (and RELENG_8)
On Tue, Aug 17, 2010 at 12:05:56PM -0700, Jack Vogel wrote: Hmmm, interesting, I'll have to have some testing done, maybe for the 574 it should automagically disable CSUM? I don't have 82574 controller to test but it may depend on how pipelined Tx data DMA works. If 82574 can still pipeline Tx data DMA when a new context is written it would be better to enable checksum offloading. If em(4) uses single Tx queue, we can safely enable checksum offloading, I guess. Jack On Tue, Aug 17, 2010 at 11:52 AM, Pyun YongHyeon pyu...@gmail.com wrote: On Mon, Aug 16, 2010 at 05:07:11PM -0400, Mike Tancsa wrote: Hi Jack, FYI, I am still seeing this same problem on RELENG_8 (code as of today). Unfortunately I cant try Pyun's patch since the underlying code has changed since then. e...@pci0:3:0:0: class=0x02 card=0x34ec8086 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c pci3: ACPI PCI bus on pcib3 em4: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x1000-0x101f mem 0xb190-0xb191,0xb192-0xb1923fff irq 16 at device 0.0 on pci3 em4: Using MSI interrupt em4: [FILTER] em4: Ethernet address: 00:15:17:ed:3e:c4 Here is updated patch for HEAD and stable/8. http://people.freebsd.org/~yongari/em.csum_tso.20100817.patchhttp://people.freebsd.org/%7Eyongari/em.csum_tso.20100817.patch It seems to work as expected under my limited environments. If you're using multiple Tx queues with em(4) it would be better to disable Tx checksum offloading as driver always have to create a new checksum context for each frame. This will effectively disable pipelined Tx data DMA which in turn greatly slows down Tx performance for small sized frames. The reason driver have to create a new checksum context when it uses multiple Tx queues comes from hardware limitation. The controller tracks only for the last context descriptor that was written such that driver does not know the state of checksum context configured in other Tx queue. Hope this helps. ---Mike At 03:36 PM 7/2/2010, Pyun YongHyeon wrote: On Fri, Jul 02, 2010 at 01:39:22PM -0400, Mike Tancsa wrote: Hi Jack, Just a followup to the email below. I now saw what appears to be the same problem on RELENG_8, but on a different nic and with VLANs. So not sure if this is a general em problem, a problem specific to some em NICs, or a TSO problem in general. The issue seemed to be triggered when I added a new vlan based on e...@pci0:14:0:0:class=0x02 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel PRO/1000 PL Network Adaptor (82573L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) pci14: ACPI PCI bus on pcib5 em3: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x6000-0x601f mem 0xe830-0xe831 irq 17 at device 0.0 on pci14 em3: Using MSI interrupt em3: [FILTER] em3: Ethernet address: 00:30:48:9f:eb:81 em3: flags=8943UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST metric 0 mtu 1500 options=2098VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC ether 00:30:48:9f:eb:81 inet 10.255.255.254 netmask 0xfffc broadcast 10.255.255.255 media: Ethernet autoselect (1000baseT full-duplex) status: active I had to disable tso, rxcsum and txsum in order to see the devices on the other side of the two vlans trunked off em3. Unfortunately, the other sides were switches 100km and 500km away so I didnt have any tcpdump capabilities to diagnose the issue. I had already created one vlan off this NIC and all was fine. A few weeks later, I added a new one and I could no longer telnet into the remote switches from the local machine But, I could telnet into the switches from machines not on the problem box. Hence, it would appear to be a general TSO issue no ? I disabled tso on the nic (I didnt disable net.inet.tcp.tso as I forgot about that).. Still nothing. I could always ping the remote devices, but no tcp services. I then remembered this issue from before, so I tried disabling tso on the NIC. Still nothing. Then I disabled rxcsum and txcsum and I could
Re: RELENG_7 em problems (and RELENG_8)
Well we do of course, i'll have my test engineer try it both ways and see what looks better. Let you know... Jack On Tue, Aug 17, 2010 at 12:35 PM, Pyun YongHyeon pyu...@gmail.com wrote: On Tue, Aug 17, 2010 at 12:05:56PM -0700, Jack Vogel wrote: Hmmm, interesting, I'll have to have some testing done, maybe for the 574 it should automagically disable CSUM? I don't have 82574 controller to test but it may depend on how pipelined Tx data DMA works. If 82574 can still pipeline Tx data DMA when a new context is written it would be better to enable checksum offloading. If em(4) uses single Tx queue, we can safely enable checksum offloading, I guess. Jack On Tue, Aug 17, 2010 at 11:52 AM, Pyun YongHyeon pyu...@gmail.com wrote: On Mon, Aug 16, 2010 at 05:07:11PM -0400, Mike Tancsa wrote: Hi Jack, FYI, I am still seeing this same problem on RELENG_8 (code as of today). Unfortunately I cant try Pyun's patch since the underlying code has changed since then. e...@pci0:3:0:0: class=0x02 card=0x34ec8086 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c pci3: ACPI PCI bus on pcib3 em4: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x1000-0x101f mem 0xb190-0xb191,0xb192-0xb1923fff irq 16 at device 0.0 on pci3 em4: Using MSI interrupt em4: [FILTER] em4: Ethernet address: 00:15:17:ed:3e:c4 Here is updated patch for HEAD and stable/8. http://people.freebsd.org/~yongari/em.csum_tso.20100817.patchhttp://people.freebsd.org/%7Eyongari/em.csum_tso.20100817.patch http://people.freebsd.org/%7Eyongari/em.csum_tso.20100817.patch It seems to work as expected under my limited environments. If you're using multiple Tx queues with em(4) it would be better to disable Tx checksum offloading as driver always have to create a new checksum context for each frame. This will effectively disable pipelined Tx data DMA which in turn greatly slows down Tx performance for small sized frames. The reason driver have to create a new checksum context when it uses multiple Tx queues comes from hardware limitation. The controller tracks only for the last context descriptor that was written such that driver does not know the state of checksum context configured in other Tx queue. Hope this helps. ---Mike At 03:36 PM 7/2/2010, Pyun YongHyeon wrote: On Fri, Jul 02, 2010 at 01:39:22PM -0400, Mike Tancsa wrote: Hi Jack, Just a followup to the email below. I now saw what appears to be the same problem on RELENG_8, but on a different nic and with VLANs. So not sure if this is a general em problem, a problem specific to some em NICs, or a TSO problem in general. The issue seemed to be triggered when I added a new vlan based on e...@pci0:14:0:0:class=0x02 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel PRO/1000 PL Network Adaptor (82573L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) pci14: ACPI PCI bus on pcib5 em3: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x6000-0x601f mem 0xe830-0xe831 irq 17 at device 0.0 on pci14 em3: Using MSI interrupt em3: [FILTER] em3: Ethernet address: 00:30:48:9f:eb:81 em3: flags=8943UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST metric 0 mtu 1500 options=2098VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC ether 00:30:48:9f:eb:81 inet 10.255.255.254 netmask 0xfffc broadcast 10.255.255.255 media: Ethernet autoselect (1000baseT full-duplex) status: active I had to disable tso, rxcsum and txsum in order to see the devices on the other side of the two vlans trunked off em3. Unfortunately, the other sides were switches 100km and 500km away so I didnt have any tcpdump capabilities to diagnose the issue. I had already created one vlan off this NIC and all was fine. A few weeks later, I added a new one and I could no longer telnet into the remote switches from the local machine But, I could telnet into the switches from machines not on the problem box. Hence, it would
Re: RELENG_7 em problems (and RELENG_8)
On Tue, Aug 17, 2010 at 12:34:31PM -0700, Jack Vogel wrote: I believe the requirement of a context descriptor for most frames in the igb driver is just the way the hardware works, I've looked over the Linux driver again and it looks like they require the same. I don't believe its a big deal, just the added descriptor for the frame. Setting up context does not cost much. The real cost comes from stopping requesting DMA for next packet whenever a new context is written. AFAIK Linux always added a new checksum context. I don't know whether Linux cares about the cost of refilling pipeline or measured the performance differences. FreeBSD noticed the difference on em(4) controllers and took appropriate action to take full advantage of the hardware feature, I think. I have to experiment how igb(4) works when it is told to reuse configured context(both checksum and TSO context). Jack On Tue, Aug 17, 2010 at 12:14 PM, Pyun YongHyeon pyu...@gmail.com wrote: On Tue, Aug 17, 2010 at 11:52:08AM -0700, Pyun YongHyeon wrote: On Mon, Aug 16, 2010 at 05:07:11PM -0400, Mike Tancsa wrote: Hi Jack, FYI, I am still seeing this same problem on RELENG_8 (code as of today). Unfortunately I cant try Pyun's patch since the underlying code has changed since then. e...@pci0:3:0:0: class=0x02 card=0x34ec8086 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c pci3: ACPI PCI bus on pcib3 em4: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x1000-0x101f mem 0xb190-0xb191,0xb192-0xb1923fff irq 16 at device 0.0 on pci3 em4: Using MSI interrupt em4: [FILTER] em4: Ethernet address: 00:15:17:ed:3e:c4 Here is updated patch for HEAD and stable/8. http://people.freebsd.org/~yongari/em.csum_tso.20100817.patchhttp://people.freebsd.org/%7Eyongari/em.csum_tso.20100817.patch It seems to work as expected under my limited environments. If you're using multiple Tx queues with em(4) it would be better to disable Tx checksum offloading as driver always have to create a new checksum context for each frame. This will effectively disable pipelined Tx data DMA which in turn greatly slows down Tx performance for small sized frames. The reason driver have to create a new checksum context when it uses multiple Tx queues comes from hardware limitation. The controller tracks only for the last context descriptor that was written such that driver does not know the state of checksum context configured in other Tx queue. Hope this helps. For igb(4) controllers, it seems we also need a way to avoid creating a new checksum context for every Tx frame as we did in em(4). Unlike em(4) controllers, igb(4) seems to maintain context per queue such that we can safely reuse previously configured context for a queue. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: RELENG_7 em problems (and RELENG_8)
The guy who worries about the Linux driver's performance is in my team, and he's a very good engineer, and we're talking about the hardware's DMA, so its not an OS issue, thus I'm not saying I'm sure, but I'm dubious about this, at least when we're talking about igb. But hey, I'm willing to be proven wrong :) Jack On Tue, Aug 17, 2010 at 12:47 PM, Pyun YongHyeon pyu...@gmail.com wrote: On Tue, Aug 17, 2010 at 12:34:31PM -0700, Jack Vogel wrote: I believe the requirement of a context descriptor for most frames in the igb driver is just the way the hardware works, I've looked over the Linux driver again and it looks like they require the same. I don't believe its a big deal, just the added descriptor for the frame. Setting up context does not cost much. The real cost comes from stopping requesting DMA for next packet whenever a new context is written. AFAIK Linux always added a new checksum context. I don't know whether Linux cares about the cost of refilling pipeline or measured the performance differences. FreeBSD noticed the difference on em(4) controllers and took appropriate action to take full advantage of the hardware feature, I think. I have to experiment how igb(4) works when it is told to reuse configured context(both checksum and TSO context). ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: RELENG_7 em problems (and RELENG_8)
At 02:52 PM 8/17/2010, Pyun YongHyeon wrote: Here is updated patch for HEAD and stable/8. http://people.freebsd.org/~yongari/em.csum_tso.20100817.patch It seems to work as expected under my limited environments. If Thanks! The patch applies cleanly and all works as expected now! I am no longer able to trigger the bug. I just use the stock unmodified driver normally, so no multi queues # vmstat -i interrupt total rate irq256: em0 149 0 irq257: em13 0 irq259: em3 971 2 irq260: ahci0 1520 3 em3: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=219bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC ether 00:15:17:xx:xx:xx inet6 fe80::215:17ff:fexx:%em3 prefixlen 64 scopeid 0x4 inet 192.168.xx.xx netmask 0xff00 broadcast 192.168.xx.xx nd6 options=3PERFORMNUD,ACCEPT_RTADV media: Ethernet autoselect (100baseTX full-duplex) status: active e...@pci0:3:0:0: class=0x02 card=0x34ec8086 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c patch em.csum_tso.20100817.patch Hmm... Looks like a unified diff to me... The text leading up to this was: -- |Index: sys/dev/e1000/if_em.c |=== |--- sys/dev/e1000/if_em.c (revision 211398) |+++ sys/dev/e1000/if_em.c (working copy) -- Patching file sys/dev/e1000/if_em.c using Plan A... Hunk #1 succeeded at 237. Hunk #2 succeeded at 1730. Hunk #3 succeeded at 1759. Hunk #4 succeeded at 1930. Hunk #5 succeeded at 3148. Hunk #6 succeeded at 3351. Hunk #7 succeeded at 3533. Hunk #8 succeeded at 3590. Hunk #9 succeeded at 3603. Hmm... The next patch looks like a unified diff to me... The text leading up to this was: -- |Index: sys/dev/e1000/if_em.h |=== |--- sys/dev/e1000/if_em.h (revision 211398) |+++ sys/dev/e1000/if_em.h (working copy) -- Patching file sys/dev/e1000/if_em.h using Plan A... Hunk #1 succeeded at 284. done ---Mike you're using multiple Tx queues with em(4) it would be better to disable Tx checksum offloading as driver always have to create a new checksum context for each frame. This will effectively disable pipelined Tx data DMA which in turn greatly slows down Tx performance for small sized frames. The reason driver have to create a new checksum context when it uses multiple Tx queues comes from hardware limitation. The controller tracks only for the last context descriptor that was written such that driver does not know the state of checksum context configured in other Tx queue. Hope this helps. ---Mike At 03:36 PM 7/2/2010, Pyun YongHyeon wrote: On Fri, Jul 02, 2010 at 01:39:22PM -0400, Mike Tancsa wrote: Hi Jack, Just a followup to the email below. I now saw what appears to be the same problem on RELENG_8, but on a different nic and with VLANs. So not sure if this is a general em problem, a problem specific to some em NICs, or a TSO problem in general. The issue seemed to be triggered when I added a new vlan based on e...@pci0:14:0:0:class=0x02 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel PRO/1000 PL Network Adaptor (82573L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) pci14: ACPI PCI bus on pcib5 em3: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x6000-0x601f mem 0xe830-0xe831 irq 17 at device 0.0 on pci14 em3: Using MSI interrupt em3: [FILTER] em3: Ethernet address: 00:30:48:9f:eb:81 em3: flags=8943UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST metric 0 mtu 1500 options=2098VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC ether 00:30:48:9f:eb:81 inet 10.255.255.254 netmask 0xfffc broadcast 10.255.255.255 media: Ethernet autoselect (1000baseT full-duplex) status: active I had to disable tso, rxcsum and txsum in order to see the devices on the other side of the two vlans trunked off em3. Unfortunately, the other
Re: RELENG_7 em problems (and RELENG_8)
On Tue, Aug 17, 2010 at 03:55:12PM -0400, Mike Tancsa wrote: At 02:52 PM 8/17/2010, Pyun YongHyeon wrote: Here is updated patch for HEAD and stable/8. http://people.freebsd.org/~yongari/em.csum_tso.20100817.patch It seems to work as expected under my limited environments. If Thanks! The patch applies cleanly and all works as expected now! I am no longer able to trigger the bug. I just use the stock unmodified driver normally, so no multi queues Glad to hear that. Thanks for testing! # vmstat -i interrupt total rate irq256: em0 149 0 irq257: em13 0 irq259: em3 971 2 irq260: ahci0 1520 3 em3: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=219bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC ether 00:15:17:xx:xx:xx inet6 fe80::215:17ff:fexx:%em3 prefixlen 64 scopeid 0x4 inet 192.168.xx.xx netmask 0xff00 broadcast 192.168.xx.xx nd6 options=3PERFORMNUD,ACCEPT_RTADV media: Ethernet autoselect (100baseTX full-duplex) status: active e...@pci0:3:0:0: class=0x02 card=0x34ec8086 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) cap 11[a0] = MSI-X supports 5 messages in map 0x1c patch em.csum_tso.20100817.patch Hmm... Looks like a unified diff to me... The text leading up to this was: -- |Index: sys/dev/e1000/if_em.c |=== |--- sys/dev/e1000/if_em.c (revision 211398) |+++ sys/dev/e1000/if_em.c (working copy) -- Patching file sys/dev/e1000/if_em.c using Plan A... Hunk #1 succeeded at 237. Hunk #2 succeeded at 1730. Hunk #3 succeeded at 1759. Hunk #4 succeeded at 1930. Hunk #5 succeeded at 3148. Hunk #6 succeeded at 3351. Hunk #7 succeeded at 3533. Hunk #8 succeeded at 3590. Hunk #9 succeeded at 3603. Hmm... The next patch looks like a unified diff to me... The text leading up to this was: -- |Index: sys/dev/e1000/if_em.h |=== |--- sys/dev/e1000/if_em.h (revision 211398) |+++ sys/dev/e1000/if_em.h (working copy) -- Patching file sys/dev/e1000/if_em.h using Plan A... Hunk #1 succeeded at 284. done ---Mike you're using multiple Tx queues with em(4) it would be better to disable Tx checksum offloading as driver always have to create a new checksum context for each frame. This will effectively disable pipelined Tx data DMA which in turn greatly slows down Tx performance for small sized frames. The reason driver have to create a new checksum context when it uses multiple Tx queues comes from hardware limitation. The controller tracks only for the last context descriptor that was written such that driver does not know the state of checksum context configured in other Tx queue. Hope this helps. ---Mike At 03:36 PM 7/2/2010, Pyun YongHyeon wrote: On Fri, Jul 02, 2010 at 01:39:22PM -0400, Mike Tancsa wrote: Hi Jack, Just a followup to the email below. I now saw what appears to be the same problem on RELENG_8, but on a different nic and with VLANs. So not sure if this is a general em problem, a problem specific to some em NICs, or a TSO problem in general. The issue seemed to be triggered when I added a new vlan based on e...@pci0:14:0:0:class=0x02 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel PRO/1000 PL Network Adaptor (82573L)' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) pci14: ACPI PCI bus on pcib5 em3: Intel(R) PRO/1000 Network Connection 7.0.5 port 0x6000-0x601f mem 0xe830-0xe831 irq 17 at device 0.0 on pci14 em3: Using MSI interrupt em3: [FILTER] em3: Ethernet address: 00:30:48:9f:eb:81 em3: flags=8943UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST metric 0 mtu 1500 options=2098VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC ether 00:30:48:9f:eb:81 inet 10.255.255.254 netmask 0xfffc broadcast 10.255.255.255 media
Soekris Sysinstall FTP NFS Failure
Hello all, I have a Soekris net5501, and need some assistance with the installation [FreeBSD 8.1]. I've configured a PXE environment with TFTP, and NFS, and I can boot and get into sysinstall. After going through the initial configuration, slices, setting the IP address (DHCP), and proceeding with an FTP installation, it fails. It's like it cannot see the ftp site. I've also tried NFS with the same problem. lq Message qqk xInstallation completed with some errors. You may wish to x xscroll through the debugging messages on VTY1 with the x xscroll-lock feature. You can also choose No at the next x xprompt and go back into the installation menus to retry x xwhichever operations have failed. x t(100%)qqu x[ OK ]x mqq[ Press enter or space ]qqj I have confirmed that FTP and NFS are both working. Since this is a headless serial connection, and I can't see what VTY1 says, what would the best way to diagnose this be? Is there a way to redirect the output somewhere? Thanks, -Ben -- Benjamin Francom Information Technology Executive http://www.benjaminfran.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Soekris Sysinstall FTP NFS Failure
On Tue, 17 Aug 2010 12:57:55 -0700 Benjamin Francom bfran...@gmail.com wrote: Hello all, I have a Soekris net5501, and need some assistance with the installation [FreeBSD 8.1]. I've configured a PXE environment with TFTP, and NFS, and I can boot and get into sysinstall. After going through the initial configuration, slices, setting the IP address (DHCP), and proceeding with an FTP installation, it fails. It's like it cannot see the ftp site. I've also tried NFS with the same problem. As far as I can tell, netinstall is broken on all recent releases. Using a static IP address seems to get further in that you don't get an immediate error, but instead you get returned to the list of FTP servers after a minute. -- Bruce Cran ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Soekris Sysinstall FTP NFS Failure
On Tue, Aug 17, 2010 at 4:53 PM, Bruce Cran br...@cran.org.uk wrote: As far as I can tell, netinstall is broken on all recent releases. Using a static IP address seems to get further in that you don't get an immediate error, but instead you get returned to the list of FTP servers after a minute. Works for me, perhaps you need to use passive ftp? -- Adam Vande More ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Soekris Sysinstall FTP NFS Failure
On 18/08/10 05:57, Benjamin Francom wrote: Hello all, I have a Soekris net5501, and need some assistance with the installation [FreeBSD 8.1]. I've configured a PXE environment with TFTP, and NFS, and I can boot and get into sysinstall. After going through the initial configuration, slices, setting the IP address (DHCP), and proceeding with an FTP installation, it fails. It's like it cannot see the ftp site. I've G'day Ben, It does work - I've done it a number of times. I presume that you've seen my guide at http://menhennitt.com.au/wordpress/2009/07/05/installing-freebsd-on-a-soekris-net5501-using-pxe-and-dnsmasq#more-14 and Jeremy's at http://jdc.parodius.com/freebsd/pxeboot_serial_install_8.html Also, there's a Soekris mailing list that often has stuff that's more Soekris specific - http://lists.soekris.com/mailman/listinfo/soekris-tech I seem to remember having the same problem that you're having at one stage, but unfortunately, I can't remember the solution. Hope this helps, Graham ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: freebsd-stable Digest, Vol 370, Issue 2
What ? You must have missed the following suggestion/request: Might also help if it was not top-posted. ;) When replying, please edit your Subject line so it is more specific than Re: Contents of freebsd-stable digest... On 08/17/2010 11:09, FOSS Deluxe wrote: Well in normal use user applications are the ones that put the load on the CPU. The kernel is pretty much lightweight in size and work when non kernel intensive tasks are at hand (like gigabit links in the networks, etc). The good news is that the kernel will have its own core to play with. On 08/17/2010 08:00 AM, freebsd-stable-requ...@freebsd.org wrote: Send freebsd-stable mailing list submissions to freebsd-stable@freebsd.org To subscribe or unsubscribe via the World Wide Web, visit http://lists.freebsd.org/mailman/listinfo/freebsd-stable or, via email, send a message with subject or body 'help' to freebsd-stable-requ...@freebsd.org You can reach the person managing the list at freebsd-stable-ow...@freebsd.org When replying, please edit your Subject line so it is more specific than Re: Contents of freebsd-stable digest... Today's Topics: 1. Re: Inconsistent IO performance (Ivan Voras) 2. Re: Inconsistent IO performance (Kevin Oberman) 3. STABLE kernel panic: privileged instruction fault (Alexey Tarasov) 4. Re: Inconsistent IO performance (Jeremy Chadwick) 5. Re: Inconsistent IO performance (Jeremy Chadwick) 6. Re: STABLE kernel panic: privileged instruction fault (Kostik Belousov) 7. Re: STABLE kernel panic: privileged instruction fault (Alexey Tarasov) 8. Re: STABLE kernel panic: privileged instruction fault (Kostik Belousov) 9. Re: STABLE kernel panic: privileged instruction fault (Alexey Tarasov) 10. Re: STABLE kernel panic: privileged instruction fault (Kostik Belousov) 11. Re: STABLE kernel panic: privileged instruction fault (Alexey Tarasov) 12. Re: RELENG_7 em problems (and RELENG_8) (Mike Tancsa) 13. Performance AMD Phenom II X6 1090T (Vladislav V. Prodan) 14. Crash in dummynet. (Pawel Tyll) 15. Re: Crash in dummynet. (Luigi Rizzo) -- jhell,v ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
