> On Oct 18, 2023, at 15:44, Hrvoje Popovski <hrv...@srce.hr> wrote: > > On 18.10.2023. 15:35, Mischa wrote: >> Hi All, >> >> Just upgraded a couple of machines to 7.4. smooth as always!! >> >> I am however seeing issues with IPv4, slowness or no throughput at all. >> The machines I have upgraded are using an Intel X540-T network card and >> is connected on 10G. >> >> ix0 at pci5 dev 0 function 0 "Intel X540T" rev 0x01, msix, 16 queues, >> address b8:ca:3a:62:ee:40 >> ix1 at pci5 dev 0 function 1 "Intel X540T" rev 0x01, msix, 16 queues, >> address b8:ca:3a:62:ee:42 >> >> root@n2:~ # sysctl kern.version >> kern.version=OpenBSD 7.4 (GENERIC.MP) #1397: Tue Oct 10 09:02:37 MDT 2023 >> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP >> >> There are a bunch of VMs running on top of it, as soon as I want to >> fetch something with ftp, for example, I don't get anything over IPv4, >> with IPv6 everything is normal. >> >> mischa@www2:~ $ ftp -4 >> https://mirror.openbsd.amsterdam/pub/OpenBSD/7.4/amd64/install74.iso >> Trying 46.23.88.18... >> Requesting >> https://mirror.openbsd.amsterdam/pub/OpenBSD/7.4/amd64/install74.iso >> 0% | | 512 KB - stalled >> -^C >> >> A trace on mirror / n2: >> >> n2:~ # tcpdump -i vport880 host 46.23.88.32 >> tcpdump: listening on vport880, link-type EN10MB >> 15:16:08.730274 www2.high5.nl.1828 > n2.high5.nl.https: S >> 2182224746:2182224746(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale >> 6,nop,nop,timestamp 2899683458 0> (DF) >> 15:16:08.730297 arp who-has www2.high5.nl tell n2.high5.nl >> 15:16:08.731535 arp reply www2.high5.nl is-at fe:51:bb:1e:12:11 >> 15:16:08.731540 n2.high5.nl.https > www2.high5.nl.1828: S >> 633749938:633749938(0) ack 2182224747 win 16384 <mss >> 1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 3129955106 >> 2899683458> (DF) >> 15:16:08.732017 www2.high5.nl.1828 > n2.high5.nl.https: . ack 1 win 256 >> <nop,nop,timestamp 2899683458 3129955106> (DF) >> 15:16:08.785752 www2.high5.nl.1828 > n2.high5.nl.https: P 1:312(311) ack >> 1 win 256 <nop,nop,timestamp 2899683510 3129955106> (DF) >> 15:16:08.786092 n2.high5.nl.https > www2.high5.nl.1828: P 1:128(127) ack >> 312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF) >> 15:16:08.786376 n2.high5.nl.https > www2.high5.nl.1828: P 128:134(6) ack >> 312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF) >> 15:16:08.786396 n2.high5.nl.https > www2.high5.nl.1828: P 134:166(32) >> ack 312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF) >> 15:16:08.786455 n2.high5.nl.https > www2.high5.nl.1828: . 166:1614(1448) >> ack 312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF) >> 15:16:08.786457 n2.high5.nl.https > www2.high5.nl.1828: . >> 1614:3062(1448) ack 312 win 271 <nop,nop,timestamp 3129955166 >> 2899683510> (DF) >> 15:16:08.786460 n2.high5.nl.https > www2.high5.nl.1828: P 3062:3803(741) >> ack 312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF) >> 15:16:08.786943 www2.high5.nl.1828 > n2.high5.nl.https: . ack 134 win >> 255 <nop,nop,timestamp 2899683510 3129955166> (DF) >> 15:16:08.796534 n2.high5.nl.https > www2.high5.nl.1828: P 3803:4345(542) >> ack 312 win 271 <nop,nop,timestamp 3129955176 2899683510> (DF) >> 15:16:08.796577 n2.high5.nl.https > www2.high5.nl.1828: P 4345:4403(58) >> ack 312 win 271 <nop,nop,timestamp 3129955176 2899683510> (DF) >> 15:16:08.797518 www2.high5.nl.1828 > n2.high5.nl.https: . ack 166 win >> 256 <nop,nop,timestamp 2899683526 3129955166,nop,nop,sack 1 {3803:4345} >>> (DF) >> 15:16:08.797522 www2.high5.nl.1828 > n2.high5.nl.https: . ack 166 win >> 256 <nop,nop,timestamp 2899683526 3129955166,nop,nop,sack 1 {3803:4403} >>> (DF) >> 15:16:09.790297 n2.high5.nl.https > www2.high5.nl.1828: . 166:1614(1448) >> ack 312 win 271 <nop,nop,timestamp 3129956176 2899683526> (DF) >> 15:16:09.790902 www2.high5.nl.1828 > n2.high5.nl.https: . ack 1614 win >> 233 <nop,nop,timestamp 2899684519 3129956176,nop,nop,sack 1 {3803:4403} >>> (DF) >> 15:16:09.790917 n2.high5.nl.https > www2.high5.nl.1828: . >> 1614:3062(1448) ack 312 win 271 <nop,nop,timestamp 3129956176 >> 2899684519> (DF) >> 15:16:09.790923 n2.high5.nl.https > www2.high5.nl.1828: P >> 3062:4403(1341) ack 312 win 271 <nop,nop,timestamp 3129956176 >> 2899684519> (DF) >> 15:16:10.790299 n2.high5.nl.https > www2.high5.nl.1828: . >> 1614:3062(1448) ack 312 win 271 <nop,nop,timestamp 3129957176 >> 2899684519> (DF) >> 15:16:10.791204 www2.high5.nl.1828 > n2.high5.nl.https: . ack 3062 win >> 233 <nop,nop,timestamp 2899685520 3129957176,nop,nop,sack 1 {3803:4403} >>> (DF) >> 15:16:10.791223 n2.high5.nl.https > www2.high5.nl.1828: P >> 3062:4403(1341) ack 312 win 271 <nop,nop,timestamp 3129957176 >> 2899685520> (DF) >> 15:16:10.791692 www2.high5.nl.1828 > n2.high5.nl.https: . ack 4403 win >> 235 <nop,nop,timestamp 2899685520 3129957176> (DF) >> 15:16:10.802647 www2.high5.nl.1828 > n2.high5.nl.https: P 312:318(6) ack >> 4403 win 256 <nop,nop,timestamp 2899685528 3129957176> (DF) >> 15:16:11.000297 n2.high5.nl.https > www2.high5.nl.1828: . ack 318 win >> 271 <nop,nop,timestamp 3129957386 2899685528> (DF) >> 15:16:11.001162 www2.high5.nl.1828 > n2.high5.nl.https: P 318:527(209) >> ack 4403 win 256 <nop,nop,timestamp 2899685730 3129957386> (DF) >> 15:16:11.001860 n2.high5.nl.https > www2.high5.nl.1828: P 4403:5059(656) >> ack 527 win 271 <nop,nop,timestamp 3129957386 2899685730> (DF) >> 15:16:11.001989 n2.high5.nl.https > www2.high5.nl.1828: . >> 5059:6507(1448) ack 527 win 271 <nop,nop,timestamp 3129957386 >> 2899685730> (DF) >> 15:16:11.001992 n2.high5.nl.https > www2.high5.nl.1828: . >> 6507:7955(1448) ack 527 win 271 <nop,nop,timestamp 3129957386 >> 2899685730> (DF) >> 15:16:11.195431 www2.high5.nl.1828 > n2.high5.nl.https: . ack 5059 win >> 256 <nop,nop,timestamp 2899685924 3129957386> (DF) >> 15:16:11.195447 n2.high5.nl.https > www2.high5.nl.1828: . >> 7955:9403(1448) ack 527 win 271 <nop,nop,timestamp 3129957576 >> 2899685924> (DF) >> >> >> Running a trace on www2 I am seeing: >> >> www2:~ # tcpdump -i vio0 host 46.23.88.18 >> tcpdump: listening on vio0, link-type EN10MB >> 15:16:08.729974 www2.high5.nl.1828 > n2.high5.nl.https: S >> 2182224746:2182224746(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale >> 6,nop,nop,timestamp 2899683458 0> (DF) >> 15:16:08.731114 arp who-has www2.high5.nl tell n2.high5.nl >> 15:16:08.731229 arp reply www2.high5.nl is-at fe:51:bb:1e:12:11 >> 15:16:08.731631 n2.high5.nl.https > www2.high5.nl.1828: S >> 633749938:633749938(0) ack 2182224747 win 16384 <mss >> 1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 3129955106 >> 2899683458> (DF) >> 15:16:08.731732 www2.high5.nl.1828 > n2.high5.nl.https: . ack 1 win 256 >> <nop,nop,timestamp 2899683458 3129955106> (DF) >> 15:16:08.785371 www2.high5.nl.1828 > n2.high5.nl.https: P 1:312(311) ack >> 1 win 256 <nop,nop,timestamp 2899683510 3129955106> (DF) >> 15:16:08.786219 n2.high5.nl.https > www2.high5.nl.1828: P 1:128(127) ack >> 312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF) >> 15:16:08.786522 n2.high5.nl.https > www2.high5.nl.1828: P 128:134(6) ack >> 312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF) >> 15:16:08.786523 n2.high5.nl.https > www2.high5.nl.1828: P 134:166(32) >> ack 312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF) >> 15:16:08.786525 truncated-ip - 1665 bytes missing!n2.high5.nl.https > >> www2.high5.nl.1828: P 166:2138(1972) ack 312 win 271 <nop,nop,timestamp >> 3129955166 2899683510> (DF) >> 15:16:08.786628 www2.high5.nl.1828 > n2.high5.nl.https: . ack 134 win >> 255 <nop,nop,timestamp 2899683510 3129955166> (DF) >> 15:16:08.797085 n2.high5.nl.https > www2.high5.nl.1828: P 3803:4345(542) >> ack 312 win 271 <nop,nop,timestamp 3129955176 2899683510> (DF) >> 15:16:08.797086 n2.high5.nl.https > www2.high5.nl.1828: P 4345:4403(58) >> ack 312 win 271 <nop,nop,timestamp 3129955176 2899683510> (DF) >> 15:16:08.797207 www2.high5.nl.1828 > n2.high5.nl.https: . ack 166 win >> 256 <nop,nop,timestamp 2899683526 3129955166,nop,nop,sack 1 {3803:4345} >>> (DF) >> 15:16:08.797208 www2.high5.nl.1828 > n2.high5.nl.https: . ack 166 win >> 256 <nop,nop,timestamp 2899683526 3129955166,nop,nop,sack 1 {3803:4403} >>> (DF) >> 15:16:09.790498 n2.high5.nl.https > www2.high5.nl.1828: . 166:1614(1448) >> ack 312 win 271 <nop,nop,timestamp 3129956176 2899683526> (DF) >> 15:16:09.790611 www2.high5.nl.1828 > n2.high5.nl.https: . ack 1614 win >> 233 <nop,nop,timestamp 2899684519 3129956176,nop,nop,sack 1 {3803:4403} >>> (DF) >> 15:16:09.791036 truncated-ip - 817 bytes missing!n2.high5.nl.https > >> www2.high5.nl.1828: P 1614:3586(1972) ack 312 win 271 <nop,nop,timestamp >> 3129956176 2899684519> (DF) >> 15:16:10.790743 n2.high5.nl.https > www2.high5.nl.1828: . >> 1614:3062(1448) ack 312 win 271 <nop,nop,timestamp 3129957176 >> 2899684519> (DF) >> 15:16:10.790872 www2.high5.nl.1828 > n2.high5.nl.https: . ack 3062 win >> 233 <nop,nop,timestamp 2899685520 3129957176,nop,nop,sack 1 {3803:4403} >>> (DF) >> 15:16:10.791287 n2.high5.nl.https > www2.high5.nl.1828: P >> 3062:4403(1341) ack 312 win 271 <nop,nop,timestamp 3129957176 >> 2899685520> (DF) >> 15:16:10.791393 www2.high5.nl.1828 > n2.high5.nl.https: . ack 4403 win >> 235 <nop,nop,timestamp 2899685520 3129957176> (DF) >> 15:16:10.802259 www2.high5.nl.1828 > n2.high5.nl.https: P 312:318(6) ack >> 4403 win 256 <nop,nop,timestamp 2899685528 3129957176> (DF) >> 15:16:11.000715 n2.high5.nl.https > www2.high5.nl.1828: . ack 318 win >> 271 <nop,nop,timestamp 3129957386 2899685528> (DF) >> 15:16:11.000830 www2.high5.nl.1828 > n2.high5.nl.https: P 318:527(209) >> ack 4403 win 256 <nop,nop,timestamp 2899685730 3129957386> (DF) >> 15:16:11.002040 n2.high5.nl.https > www2.high5.nl.1828: P 4403:5059(656) >> ack 527 win 271 <nop,nop,timestamp 3129957386 2899685730> (DF) >> 15:16:11.002042 truncated-ip - 924 bytes missing!n2.high5.nl.https > >> www2.high5.nl.1828: . 5059:7031(1972) ack 527 win 271 <nop,nop,timestamp >> 3129957386 2899685730> (DF) >> 15:16:11.195146 www2.high5.nl.1828 > n2.high5.nl.https: . ack 5059 win >> 256 <nop,nop,timestamp 2899685924 3129957386> (DF) >> 15:16:11.195567 n2.high5.nl.https > www2.high5.nl.1828: . >> 7955:9403(1448) ack 527 win 271 <nop,nop,timestamp 3129957576 >> 2899685924> (DF) >> >> >> From "What is new" I noticed: >> * Enable LRO per default in network drivers. LRO allows to receive >> aggregated packets larger than the MTU. Receiving TCP streams becomes >> much faster. Currently only ix(4) and lo(4) devices support LRO, and >> ix(4) is limited to IPv4 and hardware newer than the old 82598 model. >> >> Could this have something to do with it? >> >> Mischa > > is it better when you disable > ifconfig ix0 -tcplro > ifconfig ix1 -tcplro
I get packets! Nice! > and maybe you would need > net.inet.tcp.tso=1 Will see if that makes a difference. Thanx Hrvoje! Mischa
