Hi
I have some odd packet loss on a openbsd based router (running -current as of the beginning of september....) . The router has 6 physical interfaces (all em, Intel 82575EB), 4 of them have traffic (about 10-20 Mbps). We did some tuning (mostly with informations from: https://calomel.org/network_performance.html) and could improve the performance: Currently we use the following sysctl tweaks: sysctl kern.maxclusters=122880 sysctl net.inet.ip.ifq.maxlen=1536 sysctl net.inet.tcp.recvspace=262144 sysctl net.inet.tcp.sendspace=262144 sysctl net.inet.udp.recvspace=262144 sysctl net.inet.udp.sendspace=262144 But still we have about 1300 Ierrs per minute... When we run a simple ping, we can see that something is strange. Where the majority of packets have a rtt of 1ms or less about every tenth package shows a rtt of >250ms... I could really use a hint of what to try next (autoneg has been disabled on all interfaces for testing, now it has been enabled again...) Thank you for your inputs Andri Keller The switches on the other and of the device are both cisco 2960G with a lacp to two interfaces on the openbsd box: em0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:25:90:05:54:6c priority: 0 trunk: trunkdev trunk1 media: Ethernet autoselect (1000baseT full-duplex) status: active inet6 fe80::225:90ff:fe05:546c%em0 prefixlen 64 scopeid 0x1 em1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:25:90:05:54:6c priority: 0 trunk: trunkdev trunk1 media: Ethernet autoselect (1000baseT full-duplex) status: active inet6 fe80::225:90ff:fe05:546d%em1 prefixlen 64 scopeid 0x2 em2: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:25:90:05:54:6e priority: 0 trunk: trunkdev trunk0 media: Ethernet 1000baseT full-duplex status: active inet6 fe80::225:90ff:fe05:546e%em2 prefixlen 64 scopeid 0x3 em3: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:25:90:05:54:6e priority: 0 trunk: trunkdev trunk0 media: Ethernet autoselect (1000baseT full-duplex) status: active inet6 fe80::225:90ff:fe05:546f%em3 prefixlen 64 scopeid 0x4 trunk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:25:90:05:54:6e priority: 0 trunk: trunkproto lacp trunk id: [(8000,00:25:90:05:54:6e,4054,0000,0000), (8000,18:ef:63:bf:d7:00,0002,0000,0000)] trunkport em3 active,collecting,distributing trunkport em2 active,collecting,distributing groups: trunk media: Ethernet autoselect status: active inet ADDRESS REMOVED inet6 fe80::225:90ff:fe05:546e%trunk0 prefixlen 64 scopeid 0xa inet6 ADDRESS REMOVED trunk1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:25:90:05:54:6c priority: 0 trunk: trunkproto lacp trunk id: [(8000,00:25:90:05:54:6c,405C,0000,0000), (8000,18:ef:63:bf:d7:00,0003,0000,0000)] trunkport em1 active,collecting,distributing trunkport em0 active,collecting,distributing groups: trunk media: Ethernet autoselect status: active inet6 fe80::225:90ff:fe05:546c%trunk1 prefixlen 64 scopeid 0xb vlan56: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:25:90:05:54:6c priority: 0 vlan: 56 priority: 0 parent interface: trunk1 groups: vlan status: active inet6 fe80::225:90ff:fe05:546c%vlan56 prefixlen 64 scopeid 0x11 inet ADDRESS REMOVED netstat -m 9023 mbufs in use: 9003 mbufs allocated to data 11 mbufs allocated to packet headers 9 mbufs allocated to socket names and addresses 528/1970/512000 mbuf 2048 byte clusters in use (current/peak/max) 0/8/512000 mbuf 4096 byte clusters in use (current/peak/max) 0/8/512000 mbuf 8192 byte clusters in use (current/peak/max) 0/8/512000 mbuf 9216 byte clusters in use (current/peak/max) 0/8/512000 mbuf 12288 byte clusters in use (current/peak/max) 0/8/512000 mbuf 16384 byte clusters in use (current/peak/max) 0/8/512000 mbuf 65536 byte clusters in use (current/peak/max) 7060 Kbytes allocated to network (46% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines netstat -i | grep em Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Colls em0 1500 <Link> 00:25:90:05:54:6c 3543633259 463916 4229526062 0 0 em0 1500 fe80::%em0/ fe80::225:90ff:fe 3543633259 463916 4229526062 0 0 em1 1500 <Link> 00:25:90:05:54:6c 142491846 3140 648841540 0 0 em1 1500 fe80::%em1/ fe80::225:90ff:fe 142491846 3140 648841540 0 0 em2 1500 <Link> 00:25:90:05:54:6e 4739634837 4297634 3005824198 0 0 em2 1500 fe80::%em2/ fe80::225:90ff:fe 4739634837 4297634 3005824198 0 0 em3 1500 <Link> 00:25:90:05:54:6e 68394073 39408 595134764 0 0 em3 1500 fe80::%em3/ fe80::225:90ff:fe 68394073 39408 595134764 0 0 em4 1500 <Link> 00:30:48:fa:e9:7c 6692185 0 1380326 0 0 em4 1500 10.1.16/20 10.1.16.36 6692185 0 1380326 0 0 em4 1500 fe80::%em4/ fe80::230:48ff:fe 6692185 0 1380326 0 0 em5* 1500 <Link> 00:30:48:fa:e9:7d 0 0 0 0 0 switch config interface GigabitEthernet0/11 switchport access vlan 101 switchport mode access speed 1000 duplex full channel-group 2 mode active ! interface GigabitEthernet0/12 switchport access vlan 101 switchport mode access channel-group 2 mode active ! interface GigabitEthernet0/13 switchport trunk allowed vlan 51-54,56,170 switchport mode trunk channel-group 3 mode active ! interface GigabitEthernet0/14 switchport trunk allowed vlan 51-54,56,170 switchport mode trunk channel-group 3 mode active ! interface Port-channel2 switchport access vlan 101 switchport mode access ! interface Port-channel3 switchport trunk allowed vlan 51-54,56,170 switchport mode trunk !
