Howdy:
I have a openbsd 5.5 Release box running a busy UDP openvpn endpoint
with a 100Mbit circuit. We tunnel SIP traffic, et al, through the openvpn.
It's setup with PF, CARP, and IPSEC. We have done little to tune the OS.
A couple of months ago running 5.2 (we are since upgraded to 5.5 and we
have new hardware) we started to notice dropped segments of SIP calls and
we discovered we simultaneously are dropping UDP packets on the endpoint.
We can watch the counter netstat increase as the sip dropout occurs.
netstat -s -p udp |grep "dropped due to full socket"
345197 dropped due to full socket buffers
I'm not sure how to proceed on tuning as I read tuning via sysctl is
becoming pointless. We have pf states set to:
set limit states 100000
grep -v ^# /etc/sysctl.conf
net.inet.ip.forwarding=1 # 1=Permit forwarding (routing) of IPv4 packets
net.inet.carp.preempt=1 # 1=Enable carp(4) preemption
net.inet.carp.log=7 # log level of carp(4) info, default 2
net.inet.udp.recvspace=131028 # udp receive space
net.inet.udp.sendspace=131028 # udp send buffer
kern.bufcachepercent=90 # kernel buffer cache memory percentage
net.inet.ip.ifq.maxlen=1536
Attached are the sysctl.conf and dmesg.
OpenBSD 5.5 (GENERIC.MP) #315: Wed Mar 5 09:37:46 MST 2014
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34301018112 (32712MB)
avail mem = 33379323904 (31833MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec0f0 (38 entries)
bios0: vendor American Megatrends Inc. version "2.00" date 04/24/2014
bios0: iXsystems iX1104-813M-350
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT SSDT SSDT SSDT MCFG PRAD HPET SSDT SSDT SPMI
DMAR EINJ ERST HEST BERT
acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) PEG2(S4)
PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4)
PXSX(S4) RP05(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 3300.63 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 3300.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 3300.00 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 3300.00 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0 addr 0xf8000000, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PEG0)
acpiprt2 at acpi0: bus -1 (PEG1)
acpiprt3 at acpi0: bus -1 (PEG2)
acpiprt4 at acpi0: bus 2 (RP01)
acpiprt5 at acpi0: bus 4 (RP03)
acpiprt6 at acpi0: bus 5 (RP04)
acpiec0 at acpi0: Failed to read resource settings
acpicpu0 at acpi0: PSS
acpicpu1 at acpi0: PSS
acpicpu2 at acpi0: PSS
acpicpu3 at acpi0: PSS
acpipwrres0 at acpi0: PG00, resource for PEG0
acpipwrres1 at acpi0: PG01, resource for PEG1
acpipwrres2 at acpi0: PG02, resource for PEG2
acpipwrres3 at acpi0: FN00, resource for FAN0
acpipwrres4 at acpi0: FN01, resource for FAN1
acpipwrres5 at acpi0: FN02, resource for FAN2
acpipwrres6 at acpi0: FN03, resource for FAN3
acpipwrres7 at acpi0: FN04, resource for FAN4
acpitz0 at acpi0: critical temperature is 105 degC
acpitz1 at acpi0: critical temperature is 105 degC
acpibat0 at acpi0: BAT0 not present
acpibat1 at acpi0: BAT1 not present
acpibat2 at acpi0: BAT2 not present
acpibtn0 at acpi0: SLPB
acpibtn1 at acpi0: PWRB
acpibtn2 at acpi0: LID0
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD1F
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 3300 MHz: speeds: 3301, 3300, 3100, 2900, 2800, 2600,
2400, 2200, 2000, 1900, 1700, 1500, 1300, 1200, 1000, 800 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v3 Host" rev 0x06
ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi
pci1 at ppb0 bus 1
em0 at pci1 dev 0 function 0 "Intel I350" rev 0x01: msi, address
a0:36:9f:40:1c:e4
em1 at pci1 dev 0 function 1 "Intel I350" rev 0x01: msi, address
a0:36:9f:40:1c:e5
em2 at pci1 dev 0 function 2 "Intel I350" rev 0x01: msi, address
a0:36:9f:40:1c:e6
em3 at pci1 dev 0 function 3 "Intel I350" rev 0x01: msi, address
a0:36:9f:40:1c:e7
"Intel 8 Series xHCI" rev 0x05 at pci0 dev 20 function 0 not configured
"Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
"Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 1 not configured
ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 16
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb1 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5: msi
pci2 at ppb1 bus 2
ppb2 at pci2 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x03
pci3 at ppb2 bus 3
vga1 at pci3 dev 0 function 0 "ASPEED Technology AST2000" rev 0x30
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ppb3 at pci0 dev 28 function 2 "Intel 8 Series PCIE" rev 0xd5: msi
pci4 at ppb3 bus 4
em4 at pci4 dev 0 function 0 "Intel I210" rev 0x03: msi, address
0c:c4:7a:09:07:1a
ppb4 at pci0 dev 28 function 3 "Intel 8 Series PCIE" rev 0xd5: msi
pci5 at ppb4 bus 5
em5 at pci5 dev 0 function 0 "Intel I210" rev 0x03: msi, address
0c:c4:7a:09:07:1b
ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
pcib0 at pci0 dev 31 function 0 "Intel C224 LPC" rev 0x05
ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: msi, AHCI 1.3
scsibus0 at ahci0: 32 targets
sd0 at scsibus0 targ 0 lun 0: <ATA, Samsung SSD 840, DXM0> SCSI3 0/direct fixed
naa.50025385a022cded
sd0: 122104MB, 512 bytes/sector, 250069680 sectors, thin
sd1 at scsibus0 targ 1 lun 0: <ATA, Samsung SSD 840, DXM0> SCSI3 0/direct fixed
naa.50025385a022cdf8
sd1: 122104MB, 512 bytes/sector, 250069680 sectors, thin
ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x05: apic 8 int 18
iic0 at ichiic0
sdtemp0 at iic0 addr 0x18: stts2002
sdtemp1 at iic0 addr 0x19: stts2002
sdtemp2 at iic0 addr 0x1a: stts2002
sdtemp3 at iic0 addr 0x1b: stts2002
spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem1 at iic0 addr 0x51: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem2 at iic0 addr 0x52: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem3 at iic0 addr 0x53: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
"Intel 8 Series Thermal" rev 0x05 at pci0 dev 31 function 6 not configured
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
com2 at isa0 port 0x3e8/8 irq 5: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
wbsio0 at isa0 port 0x2e/2: NCT6776F rev 0x33
lm1 at wbsio0 port 0xa30/8: NCT6776F
uhub2 at uhub0 port 1 "Intel Rate Matching Hub" rev 2.00/0.05 addr 2
uhub3 at uhub2 port 3 "vendor 0x0000 product 0x0001" rev 2.00/0.00 addr 3
uhidev0 at uhub3 port 1 configuration 1 interface 0 "ATEN International product
0x2419" rev 1.10/1.00 addr 4
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0: console keyboard, using wsdisplay0
uhidev1 at uhub3 port 1 configuration 1 interface 1 "ATEN International product
0x2419" rev 1.10/1.00 addr 4
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhub4 at uhub1 port 1 "Intel Rate Matching Hub" rev 2.00/0.05 addr 2
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
sd2 at scsibus2 targ 1 lun 0: <OPENBSD, SR RAID 1, 005> SCSI2 0/direct fixed
sd2: 121969MB, 512 bytes/sector, 249794174 sectors
root on sd2a (245667dcf1c0d0e6.a) swap on sd2b dump on sd2b
Thanks in advance.
-dkw
