Re: Intel i350 Offloading not working
All: More than anything what I wanted was a justification for why things were; not sure why you took that as complaining Theo. At the same time Theo, Claudio your last replies have been very informative as far as the bugs hw offloading introduces etc etc and the reasoning. Thanks all! -- Adonis > On Jul 21, 2018, at 3:59 PM, Theo de Raadt wrote: > > Claudio Jeker wrote: > >>> On Sat, Jul 21, 2018 at 07:02:08PM +, Stuart Henderson wrote: On 2018-07-21, Adonis Peralta wrote: Is there a reason why the offloading features shouldn???t work correctly on OpenBSD? >>> >>> If you can figure out why it doesn't work, you'll be well on the way to >>> fixing it. >>> i350 supports offloading just fine via the igb driver on FreeBSD. Is it more work on the driver thats needed? >>> >>> Different OS, different driver.. >>> >>> No idea if it's the case with i350 too, but checksum offload has been >>> rather underwhelming when supported on some other NICs. >> >> We had to turn of many of the HW features used by other OS becuase of HW >> bugs. em(4) is not an exception. Lately it has become more obvious that >> most of those offload features are either bad for you or the gain does not >> really justify the effort. The amount of bugs we hit because of such >> features are countless. NFS and multicast packets are just two things to >> mention which broke on Intel cards when enabling some of the offloading >> features. I lost interest in offloading since CPUs are now fast enough. > > My favorite hw offloading bug was in the Tehuti 10G ethernet, which > would perform ipv4 checksumming against non-ipv4 packets. Fantastic > fun. > > When hardware has a feature, it doesn't mean require you to use it. You > need to JUSTIFY the need. Just recently, we failed to support the intel > debug registers, we hadn't felt the yearning to support a feature other > architectures don't have an exact similar feature for. As a result > everyone else had the pop ss bug, but we avoided it. > > It's science. So Adonis, steps for you are: 1) Stop complaining. > 2) JUSTIFY the need. 3) Do the work. End of story. >
Re: Intel i350 Offloading not working
Claudio Jeker wrote: > On Sat, Jul 21, 2018 at 07:02:08PM +, Stuart Henderson wrote: > > On 2018-07-21, Adonis Peralta wrote: > > > Is there a reason why the offloading features shouldn???t work correctly > > > on OpenBSD? > > > > If you can figure out why it doesn't work, you'll be well on the way to > > fixing it. > > > > > i350 supports offloading just fine via the igb driver on FreeBSD. Is > > > it more work on the driver thats needed? > > > > Different OS, different driver.. > > > > No idea if it's the case with i350 too, but checksum offload has been > > rather underwhelming when supported on some other NICs. > > We had to turn of many of the HW features used by other OS becuase of HW > bugs. em(4) is not an exception. Lately it has become more obvious that > most of those offload features are either bad for you or the gain does not > really justify the effort. The amount of bugs we hit because of such > features are countless. NFS and multicast packets are just two things to > mention which broke on Intel cards when enabling some of the offloading > features. I lost interest in offloading since CPUs are now fast enough. My favorite hw offloading bug was in the Tehuti 10G ethernet, which would perform ipv4 checksumming against non-ipv4 packets. Fantastic fun. When hardware has a feature, it doesn't mean require you to use it. You need to JUSTIFY the need. Just recently, we failed to support the intel debug registers, we hadn't felt the yearning to support a feature other architectures don't have an exact similar feature for. As a result everyone else had the pop ss bug, but we avoided it. It's science. So Adonis, steps for you are: 1) Stop complaining. 2) JUSTIFY the need. 3) Do the work. End of story.
Re: Intel i350 Offloading not working
On Sat, Jul 21, 2018 at 07:02:08PM +, Stuart Henderson wrote: > On 2018-07-21, Adonis Peralta wrote: > > Is there a reason why the offloading features shouldn???t work correctly > > on OpenBSD? > > If you can figure out why it doesn't work, you'll be well on the way to > fixing it. > > > i350 supports offloading just fine via the igb driver on FreeBSD. Is > > it more work on the driver thats needed? > > Different OS, different driver.. > > No idea if it's the case with i350 too, but checksum offload has been > rather underwhelming when supported on some other NICs. We had to turn of many of the HW features used by other OS becuase of HW bugs. em(4) is not an exception. Lately it has become more obvious that most of those offload features are either bad for you or the gain does not really justify the effort. The amount of bugs we hit because of such features are countless. NFS and multicast packets are just two things to mention which broke on Intel cards when enabling some of the offloading features. I lost interest in offloading since CPUs are now fast enough. -- :wq Claudio
Re: Intel i350 Offloading not working
On 2018-07-21, Adonis Peralta wrote: > Is there a reason why the offloading features shouldn’t work correctly > on OpenBSD? If you can figure out why it doesn't work, you'll be well on the way to fixing it. > i350 supports offloading just fine via the igb driver on FreeBSD. Is > it more work on the driver thats needed? Different OS, different driver.. No idea if it's the case with i350 too, but checksum offload has been rather underwhelming when supported on some other NICs.
Re: Intel i350 Offloading not working
Uh? Because? *laughs* Hi Theo. -- Adonis On Jul 21, 2018, at 1:24 PM, Theo de Raadt wrote: Hi Adonis, Because. Adonis Peralta wrote: Is there a reason why the offloading features shouldn’t work correctly on OpenBSD? i350 supports offloading just fine via the igb driver on FreeBSD. Is it more work on the driver thats needed? -- Adonis On Jul 18, 2018, at 3:39 PM, Stuart Henderson wrote: On 2018-07-18, Adonis Peralta wrote: At the same time it seems that the documentation on em(4) is incorrect for stating i350 supports offloading currently. Yes, I think this is just a manpage omission at this point. Because the driver *explicitly* disables offload for this chip it's pretty unlikely to actually work without making other changes to the driver.
Re: Intel i350 Offloading not working
Hi Adonis, Because. Adonis Peralta wrote: > Is there a reason why the offloading features shouldn???t work correctly on > OpenBSD? i350 supports offloading just fine via the igb driver on FreeBSD. Is > it more work on the driver thats needed? > > -- > Adonis > > > On Jul 18, 2018, at 3:39 PM, Stuart Henderson wrote: > > > >> On 2018-07-18, Adonis Peralta wrote: > >> At the same time it seems that the documentation on em(4) is incorrect for > >> stating i350 supports offloading currently. > > > > Yes, I think this is just a manpage omission at this point. Because the > > driver *explicitly* disables offload for this chip it's pretty unlikely > > to actually work without making other changes to the driver. > > > > >
Re: Intel i350 Offloading not working
Is there a reason why the offloading features shouldn’t work correctly on OpenBSD? i350 supports offloading just fine via the igb driver on FreeBSD. Is it more work on the driver thats needed? -- Adonis > On Jul 18, 2018, at 3:39 PM, Stuart Henderson wrote: > >> On 2018-07-18, Adonis Peralta wrote: >> At the same time it seems that the documentation on em(4) is incorrect for >> stating i350 supports offloading currently. > > Yes, I think this is just a manpage omission at this point. Because the > driver *explicitly* disables offload for this chip it's pretty unlikely > to actually work without making other changes to the driver. > >
Re: Intel i350 Offloading not working
On 2018-07-18, Adonis Peralta wrote: > At the same time it seems that the documentation on em(4) is incorrect for > stating i350 supports offloading currently. Yes, I think this is just a manpage omission at this point. Because the driver *explicitly* disables offload for this chip it's pretty unlikely to actually work without making other changes to the driver.
Re: Intel i350 Offloading not working
At the same time it seems that the documentation on em(4) is incorrect for stating i350 supports offloading currently. — Adonis >>> On Jul 18, 2018, at 6:58 AM, Kim Zeitler wrote: >>> >>> On 07/18/18 11:37, Adonis Peralta wrote: >>> Will definitely do that, but still looking for any explanation from devs :). >> https://marc.info/?l=openbsd-tech=135203532704213=2 >> >> Seems there have been some errors with offloading and I350 in the past >> >> Cheers >> Kim >>
Re: Intel i350 Offloading not working
On 07/18/18 11:37, Adonis Peralta wrote: Will definitely do that, but still looking for any explanation from devs :). https://marc.info/?l=openbsd-tech=135203532704213=2 Seems there have been some errors with offloading and I350 in the past Cheers Kim smime.p7s Description: S/MIME Cryptographic Signature
Re: Intel i350 Offloading not working
Will definitely do that, but still looking for any explanation from devs :). -- Adonis > On Jul 18, 2018, at 5:28 AM, LÉVAI Dániel wrote: > > Adonis Peralta @ 2018-07-18T10:49:57 +0200: >>> Maybe this is the culprit? >>> >>> /usr/src/sys/dev/pci/if_em.c: >>> 1893 if (sc->hw.mac_type >= em_82543 && sc->hw.mac_type != >>> em_82575 && >>> 1894 sc->hw.mac_type != em_82580 && sc->hw.mac_type != >>> em_i210 && >>> 1895 sc->hw.mac_type != em_i350) >>> 1896 ifp->if_capabilities |= IFCAP_CSUM_TCPv4 | >>> IFCAP_CSUM_UDPv4; >>> >>> It seems as if I350 (among a few other) has an exception for these >>> features. Maybe it was not tested well enough, or actually it was, and >>> was failing in this department. >>> Maybe a developer can add some insight, if this catches their eyes :) >>> >> Exactly! This is a really really nice card and I don’t get why OpenBSD >> doesn’t support the offloading features just as you’ve shown for bge and >> re0. Definitely awaiting a response from the devs on this. > > If I'm not mistaken this was introduced with this commit: > http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/dev/pci/if_em.c.diff?r1=1.269=1.270 > > Without trying to sound like the chicken [1], if you're adventurous enough, > you can always enable it for I350 and > 1) recompile > 2) try > 3) report back / send in diff :) > > > Daniel > > [1] https://en.wikipedia.org/wiki/The_Chicken_and_the_Pig > > -- > LÉVAI Dániel > PGP key ID = 0x83B63A8F > Key fingerprint = DBEC C66B A47A DFA2 792D 650C C69B BE4C 83B6 3A8F >
Re: Intel i350 Offloading not working
Adonis Peralta @ 2018-07-18T10:49:57 +0200: > > Maybe this is the culprit? > > > > /usr/src/sys/dev/pci/if_em.c: > > 1893 if (sc->hw.mac_type >= em_82543 && sc->hw.mac_type != > > em_82575 && > > 1894 sc->hw.mac_type != em_82580 && sc->hw.mac_type != > > em_i210 && > > 1895 sc->hw.mac_type != em_i350) > > 1896 ifp->if_capabilities |= IFCAP_CSUM_TCPv4 | > > IFCAP_CSUM_UDPv4; > > > > It seems as if I350 (among a few other) has an exception for these > > features. Maybe it was not tested well enough, or actually it was, and > > was failing in this department. > > Maybe a developer can add some insight, if this catches their eyes :) > > > Exactly! This is a really really nice card and I don’t get why OpenBSD > doesn’t support the offloading features just as you’ve shown for bge and > re0. Definitely awaiting a response from the devs on this. If I'm not mistaken this was introduced with this commit: http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/dev/pci/if_em.c.diff?r1=1.269=1.270 Without trying to sound like the chicken [1], if you're adventurous enough, you can always enable it for I350 and 1) recompile 2) try 3) report back / send in diff :) Daniel [1] https://en.wikipedia.org/wiki/The_Chicken_and_the_Pig -- LÉVAI Dániel PGP key ID = 0x83B63A8F Key fingerprint = DBEC C66B A47A DFA2 792D 650C C69B BE4C 83B6 3A8F
Re: Intel i350 Offloading not working
Exactly! This is a really really nice card and I don’t get why OpenBSD doesn’t support the offloading features just as you’ve shown for bge and re0. Definitely awaiting a response from the devs on this. -- Adonis On Jul 18, 2018, at 4:34 AM, LÉVAI Dániel wrote: LÉVAI Dániel @ 2018-07-18T09:20:15 +0200: Adonis Peralta @ 2018-07-18T03:47:43 +0200: Hi, [...] ifconfig on my lan port shows: ``` em2: flags=8843 mtu 1500 hwfeatures=10 hardmtu 9216 lladdr 00:19:99:d7:88:a3 index 3 priority 0 llprio 3 media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause) status: active inet 192.168.1.1 netmask 0xff00 broadcast 192.168.1.255 ``` [...] # ifconfig em0 hwfeatures em0: flags=8843 mtu 1500 hwfeatures=10 hardmtu 9216 [...] Maybe this is the culprit? /usr/src/sys/dev/pci/if_em.c: 1893 if (sc->hw.mac_type >= em_82543 && sc->hw.mac_type != em_82575 && 1894 sc->hw.mac_type != em_82580 && sc->hw.mac_type != em_i210 && 1895 sc->hw.mac_type != em_i350) 1896 ifp->if_capabilities |= IFCAP_CSUM_TCPv4 | IFCAP_CSUM_UDPv4; It seems as if I350 (among a few other) has an exception for these features. Maybe it was not tested well enough, or actually it was, and was failing in this department. Maybe a developer can add some insight, if this catches their eyes :) Daniel -- LÉVAI Dániel PGP key ID = 0x83B63A8F Key fingerprint = DBEC C66B A47A DFA2 792D 650C C69B BE4C 83B6 3A8F
Re: Intel i350 Offloading not working
LÉVAI Dániel @ 2018-07-18T09:20:15 +0200: > Adonis Peralta @ 2018-07-18T03:47:43 +0200: > > Hi, > > [...] > > ifconfig on my lan port shows: > > > > ``` > > em2: flags=8843 mtu 1500 > > hwfeatures=10 hardmtu 9216 > > lladdr 00:19:99:d7:88:a3 > > index 3 priority 0 llprio 3 > > media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause) > > status: active > > inet 192.168.1.1 netmask 0xff00 broadcast 192.168.1.255 > > ``` [...] > # ifconfig em0 hwfeatures > em0: flags=8843 mtu 1500 > hwfeatures=10 hardmtu 9216 > [...] Maybe this is the culprit? /usr/src/sys/dev/pci/if_em.c: 1893 if (sc->hw.mac_type >= em_82543 && sc->hw.mac_type != em_82575 && 1894 sc->hw.mac_type != em_82580 && sc->hw.mac_type != em_i210 && 1895 sc->hw.mac_type != em_i350) 1896 ifp->if_capabilities |= IFCAP_CSUM_TCPv4 | IFCAP_CSUM_UDPv4; It seems as if I350 (among a few other) has an exception for these features. Maybe it was not tested well enough, or actually it was, and was failing in this department. Maybe a developer can add some insight, if this catches their eyes :) Daniel -- LÉVAI Dániel PGP key ID = 0x83B63A8F Key fingerprint = DBEC C66B A47A DFA2 792D 650C C69B BE4C 83B6 3A8F
Re: Intel i350 Offloading not working
Adonis Peralta @ 2018-07-18T03:47:43 +0200: > Hi, > > I am running OpenBSD 6.3 with a PCI-Express Intel i350T4 Network card. The > card is detected with the em(4) driver but upon looking at the hwfeatures > with ifconfig I notice that none of the offloading features the card > supports are enabled. Looking at the documentation for the em(4) driver it > states that it supports most of the offloading features: > > ``` > “The em driver supports IPv4 receive IP/TCP/UDP checksum offload and > transmit TCP/UDP checksum offload on all but > 82542-based adapters, VLAN tag insertion and stripping, and jumbo frames on > all but 82562V, 82566DC/82566DM > and 82573E/82573L/82573V-based adapters. The 82562V Ethernet controller chip > only supports 10/100 media types.” > ``` > > Yet none are enabled. Any ideas how to get them enabled on OpenBSD? > > My pertinent dmesg(8) output related to the i350: > > ``` > em1 at pci19 dev 0 function 0 "Intel I350" rev 0x01: apic 2 int 16, address > 00:19:99:d7:88:a2 > em2 at pci19 dev 0 function 1 "Intel I350" rev 0x01: apic 2 int 17, address > 00:19:99:d7:88:a3 > em3 at pci19 dev 0 function 2 "Intel I350" rev 0x01: apic 2 int 18, address > 00:19:99:d7:88:a4 > em4 at pci19 dev 0 function 3 "Intel I350" rev 0x01: apic 2 int 19, address > 00:19:99:d7:88:a5 > ``` > > ifconfig on my lan port shows: > > ``` > em2: flags=8843 mtu 1500 > hwfeatures=10 hardmtu 9216 > lladdr 00:19:99:d7:88:a3 > index 3 priority 0 llprio 3 > media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause) > status: active > inet 192.168.1.1 netmask 0xff00 broadcast 192.168.1.255 > ``` Interesting, this post got my attention as I have the same quad-port card, and never actually bothered to look at the hw features. I have the same output: em0 at pci1 dev 0 function 0 "Intel I350" rev 0x01: msi, address em1 at pci1 dev 0 function 1 "Intel I350" rev 0x01: msi, address em2 at pci1 dev 0 function 2 "Intel I350" rev 0x01: msi, address em3 at pci1 dev 0 function 3 "Intel I350" rev 0x01: msi, address # ifconfig em0 hwfeatures em0: flags=8843 mtu 1500 hwfeatures=10 hardmtu 9216 [...] In comparison, bge0, and re0(APU2): bge0: flags=208b43 mtu 1500 hwfeatures=33 hardmtu 1500 [...] re0: flags=8843 mtu 1500 hwfeatures=8037 hardmtu 9194 [...] I guess this somewhat is what you were looking for in em's output? Daniel dmesg: OpenBSD 6.3 (GENERIC.MP) #4: Sun Jun 17 11:22:20 CEST 2018 r...@syspatch-63-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 2059997184 (1964MB) avail mem = 1990483968 (1898MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec130 (75 entries) bios0: vendor American Megatrends Inc. version "F2" date 08/06/2015 bios0: Gigabyte Technology Co., Ltd. H81M-DS2 acpi0 at bios0: rev 2 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP APIC FPDT SSDT SSDT MCFG HPET SSDT SSDT acpi0: wakeup devices PXSX(S4) RP01(S4) PXSX(S4) PXSX(S4) RP03(S4) PXSX(S4) PXSX(S4) RP05(S4) PXSX(S4) RP06(S4) PXSX(S4) PXSX(S4) GLAN(S4) EHC1(S4) EHC2(S4) XHC_(S4) [...] acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Celeron(R) CPU G1840 @ 2.80GHz, 2793.94 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,ERMS,INVPCID,IBRS,IBPB,STIBP,SENSOR,ARAT,XSAVEOPT,MELTDOWN cpu0: 256KB 64b/line 8-way L2 cache acpitimer0: recalibrated TSC frequency 2793534545 Hz cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges cpu0: apic clock running at 99MHz cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE cpu1 at mainbus0: apid 2 (application processor) cpu1: Intel(R) Celeron(R) CPU G1840 @ 2.80GHz, 2793.54 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,ERMS,INVPCID,IBRS,IBPB,STIBP,SENSOR,ARAT,XSAVEOPT,MELTDOWN cpu1: 256KB 64b/line 8-way L2 cache cpu1: smt 0, core 1, package 0 ioapic0 at mainbus0: apid 8 pa 0xfec0, version 20, 24 pins acpimcfg0 at acpi0 addr 0xf800, bus 0-63 acpihpet0 at acpi0: 14318179 Hz acpihpet0: recalibrated TSC frequency 2793551440 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 2 (RP01) acpiprt2 at acpi0: bus 3 (RP03) acpiprt3 at acpi0: bus 4 (RP05) acpiprt4 at acpi0: bus 5 (RP06) acpiprt5 at acpi0: bus 1 (PEG0) acpiprt6 at acpi0: bus -1 (PEG1) acpiprt7 at acpi0: bus -1 (PEG2) acpiec0 at
