On Mon, 4 Mar 2019, 13:29 David Gwynne, <da...@gwynne.id.au> wrote: > On Mon, Mar 04, 2019 at 10:36:23AM +0100, Tony Sarendal wrote: > > On Mon, 4 Mar 2019, 09:43 Tony Sarendal, <t...@polarcap.org> wrote: > > > > > > > > > > > Den m??n 4 mars 2019 kl 09:26 skrev Tony Sarendal <t...@polarcap.org>: > > > > > >> Den s??n 3 mars 2019 kl 21:35 skrev Theo de Raadt < > dera...@openbsd.org>: > > >> > > >>> Tony, > > >>> > > >>> Are you out of your mind? You didn't provide even a rough hint about > > >>> what your firewall configuration looks like. You recognize that's > > >>> pathetic, right? > > >>> > > >>> > Earlier in the week I could run parallel ping-pong tests through my > > >>> test > > >>> > firewalls > > >>> > at 300kpps without any packet loss. I updated to the latest > snapshot > > >>> today > > >>> > and > > >>> > start to see packet loss at around 80kpps. > > >>> > > > >>> > /T > > >>> > > > >>> > OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar 3 10:24:08 MST 2019 > > >>> > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/ > > >>> GENERIC.MP > > >>> > real mem = 34300891136 (32711MB) > > >>> > avail mem = 33251393536 (31711MB) > > >>> > mpath0 at root > > >>> > scsibus0 at mpath0: 256 targets > > >>> > mainbus0 at root > > >>> > bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries) > > >>> > bios0: vendor American Megatrends Inc. version "3.0" date > 04/24/2015 > > >>> > bios0: Supermicro X10SLD > > >>> > acpi0 at bios0: rev 2 > > >>> > acpi0: sleep states S0 S4 S5 > > >>> > acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT MCFG PRAD HPET > SSDT > > >>> SSDT > > >>> > SPMI DMAR EINJ ERST HEST BERT > > >>> > acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) > > >>> PEG2(S4) > > >>> > PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) > RP04(S4) > > >>> > PXSX(S4) RP05(S4) [...] > > >>> > acpitimer0 at acpi0: 3579545 Hz, 24 bits > > >>> > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat > > >>> > cpu0 at mainbus0: apid 0 (boot processor) > > >>> > cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.68 MHz, > 06-3c-03 > > >>> > cpu0: > > >>> > > > >>> > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > > >>> > cpu0: 256KB 64b/line 8-way L2 cache > > >>> > cpu0: smt 0, core 0, package 0 > > >>> > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges > > >>> > cpu0: apic clock running at 99MHz > > >>> > cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE > > >>> > cpu1 at mainbus0: apid 2 (application processor) > > >>> > cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, > 06-3c-03 > > >>> > cpu1: > > >>> > > > >>> > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > > >>> > cpu1: 256KB 64b/line 8-way L2 cache > > >>> > cpu1: smt 0, core 1, package 0 > > >>> > cpu2 at mainbus0: apid 4 (application processor) > > >>> > cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, > 06-3c-03 > > >>> > cpu2: > > >>> > > > >>> > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > > >>> > cpu2: 256KB 64b/line 8-way L2 cache > > >>> > cpu2: smt 0, core 2, package 0 > > >>> > cpu3 at mainbus0: apid 6 (application processor) > > >>> > cpu3: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, > 06-3c-03 > > >>> > cpu3: > > >>> > > > >>> > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > > >>> > cpu3: 256KB 64b/line 8-way L2 cache > > >>> > cpu3: smt 0, core 3, package 0 > > >>> > ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins > > >>> > acpimcfg0 at acpi0 > > >>> > acpimcfg0: addr 0xf8000000, bus 0-63 > > >>> > acpihpet0 at acpi0: 14318179 Hz > > >>> > acpiprt0 at acpi0: bus 0 (PCI0) > > >>> > acpiprt1 at acpi0: bus 1 (PEG0) > > >>> > acpiprt2 at acpi0: bus 2 (PEG1) > > >>> > acpiprt3 at acpi0: bus -1 (PEG2) > > >>> > acpiprt4 at acpi0: bus 3 (RP01) > > >>> > acpiprt5 at acpi0: bus -1 (RP02) > > >>> > acpiprt6 at acpi0: bus -1 (RP03) > > >>> > acpiprt7 at acpi0: bus -1 (RP04) > > >>> > acpiprt8 at acpi0: bus -1 (RP05) > > >>> > acpiprt9 at acpi0: bus -1 (RP06) > > >>> > acpiprt10 at acpi0: bus -1 (RP07) > > >>> > acpiprt11 at acpi0: bus -1 (RP08) > > >>> > acpiec0 at acpi0: not present > > >>> > acpicpu0 at acpi0: C1(@1 halt!) > > >>> > acpicpu1 at acpi0: C1(@1 halt!) > > >>> > acpicpu2 at acpi0: C1(@1 halt!) > > >>> > acpicpu3 at acpi0: C1(@1 halt!) > > >>> > acpipwrres0 at acpi0: PG00, resource for PEG0 > > >>> > acpipwrres1 at acpi0: PG01, resource for PEG1 > > >>> > acpipwrres2 at acpi0: PG02, resource for PEG2 > > >>> > acpipwrres3 at acpi0: FN00, resource for FAN0 > > >>> > acpipwrres4 at acpi0: FN01, resource for FAN1 > > >>> > acpipwrres5 at acpi0: FN02, resource for FAN2 > > >>> > acpipwrres6 at acpi0: FN03, resource for FAN3 > > >>> > acpipwrres7 at acpi0: FN04, resource for FAN4 > > >>> > acpitz0 at acpi0: critical temperature is 105 degC > > >>> > acpitz1 at acpi0: critical temperature is 105 degC > > >>> > acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001 > > >>> > acpicmos0 at acpi0 > > >>> > "IPI0001" at acpi0 not configured > > >>> > acpibtn0 at acpi0: SLPB > > >>> > acpibtn1 at acpi0: PWRB > > >>> > "PNP0C0B" at acpi0 not configured > > >>> > "PNP0C0B" at acpi0 not configured > > >>> > "PNP0C0B" at acpi0 not configured > > >>> > "PNP0C0B" at acpi0 not configured > > >>> > "PNP0C0B" at acpi0 not configured > > >>> > acpivideo0 at acpi0: GFX0 > > >>> > acpivout0 at acpivideo0: DD1F > > >>> > ipmi at mainbus0 not configured > > >>> > pci0 at mainbus0 bus 0 > > >>> > pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v3 Host" rev > 0x06 > > >>> > ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi > > >>> > pci1 at ppb0 bus 1 > > >>> > ix0 at pci1 dev 0 function 0 "Intel 82599" rev 0x01: msi, address > > >>> > 0c:c4:7a:59:30:a4 > > >>> > ix1 at pci1 dev 0 function 1 "Intel 82599" rev 0x01: msi, address > > >>> > 0c:c4:7a:59:30:a5 > > >>> > ppb1 at pci0 dev 1 function 1 "Intel Core 4G PCIE" rev 0x06: msi > > >>> > pci2 at ppb1 bus 2 > > >>> > ixl0 at pci2 dev 0 function 0 "Intel XL710 QSFP+" rev 0x02: port > 0, FW > > >>> > 5.0.40043 API 1.5, msi, address 0c:c4:7a:5e:f9:c8 > > >>> > ixl0: unable to query phy types > > >>> > ixl1 at pci2 dev 0 function 1 "Intel XL710 QSFP+" rev 0x02: port > 1, FW > > >>> > 5.0.40043 API 1.5, msi, address 0c:c4:7a:5e:f9:c9 > > >>> > ixl1: unable to query phy types > > >>> > "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not > configured > > >>> > "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 1 not > configured > > >>> > ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x05: > apic 8 > > >>> int 16 > > >>> > usb0 at ehci0: USB revision 2.0 > > >>> > uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev > > >>> > 2.00/1.00 addr 1 > > >>> > ppb2 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5: msi > > >>> > pci3 at ppb2 bus 3 > > >>> > ppb3 at pci3 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev > 0x03 > > >>> > pci4 at ppb3 bus 4 > > >>> > vga1 at pci4 dev 0 function 0 "ASPEED Technology AST2000" rev 0x30 > > >>> > wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) > > >>> > wsdisplay0: screen 1-5 added (80x25, vt100 emulation) > > >>> > ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x05: > apic 8 > > >>> int 22 > > >>> > usb1 at ehci1: USB revision 2.0 > > >>> > uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev > > >>> > 2.00/1.00 addr 1 > > >>> > pcib0 at pci0 dev 31 function 0 "Intel C224 LPC" rev 0x05 > > >>> > ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: > msi, > > >>> AHCI > > >>> > 1.3 > > >>> > ahci0: port 0: 6.0Gb/s > > >>> > ahci0: port 1: 6.0Gb/s > > >>> > scsibus1 at ahci0: 32 targets > > >>> > sd0 at scsibus1 targ 0 lun 0: <ATA, INTEL SSDSC2BB12, G201> SCSI3 > > >>> 0/direct > > >>> > fixed naa.55cd2e404c022db4 > > >>> > sd0: 114473MB, 512 bytes/sector, 234441648 sectors, thin > > >>> > sd1 at scsibus1 targ 1 lun 0: <ATA, INTEL SSDSC2BB12, G201> SCSI3 > > >>> 0/direct > > >>> > fixed naa.55cd2e404c022518 > > >>> > sd1: 114473MB, 512 bytes/sector, 234441648 sectors, thin > > >>> > ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x05: > > >>> apic 8 > > >>> > int 18 > > >>> > iic0 at ichiic0 > > >>> > sdtemp0 at iic0 addr 0x18: stts2002 > > >>> > sdtemp1 at iic0 addr 0x19: stts2002 > > >>> > sdtemp2 at iic0 addr 0x1a: stts2002 > > >>> > sdtemp3 at iic0 addr 0x1b: stts2002 > > >>> > spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM ECC PC3-12800 with > thermal > > >>> sensor > > >>> > spdmem1 at iic0 addr 0x51: 8GB DDR3 SDRAM ECC PC3-12800 with > thermal > > >>> sensor > > >>> > spdmem2 at iic0 addr 0x52: 8GB DDR3 SDRAM ECC PC3-12800 with > thermal > > >>> sensor > > >>> > spdmem3 at iic0 addr 0x53: 8GB DDR3 SDRAM ECC PC3-12800 with > thermal > > >>> sensor > > >>> > "Intel 8 Series Thermal" rev 0x05 at pci0 dev 31 function 6 not > > >>> configured > > >>> > isa0 at pcib0 > > >>> > isadma0 at isa0 > > >>> > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo > > >>> > com2 at isa0 port 0x3e8/8 irq 5: ns16550a, 16 byte fifo > > >>> > pckbc0 at isa0 port 0x60/5 irq 1 irq 12 > > >>> > pcppi0 at isa0 port 0x61 > > >>> > spkr0 at pcppi0 > > >>> > wbsio0 at isa0 port 0x2e/2: NCT6776F rev 0x33 > > >>> > lm1 at wbsio0 port 0xa30/8: NCT6776F > > >>> > vmm0 at mainbus0: VMX/EPT > > >>> > uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate > Matching > > >>> Hub" > > >>> > rev 2.00/0.05 addr 2 > > >>> > uhub3 at uhub2 port 3 configuration 1 interface 0 "ATEN > International > > >>> > product 0x7000" rev 2.00/0.00 addr 3 > > >>> > uhidev0 at uhub3 port 1 configuration 1 interface 0 "ATEN > International > > >>> > product 0x2419" rev 1.10/1.00 addr 4 > > >>> > uhidev0: iclass 3/1 > > >>> > ukbd0 at uhidev0: 8 variable keys, 6 key codes > > >>> > wskbd0 at ukbd0: console keyboard, using wsdisplay0 > > >>> > uhidev1 at uhub3 port 1 configuration 1 interface 1 "ATEN > International > > >>> > product 0x2419" rev 1.10/1.00 addr 4 > > >>> > uhidev1: iclass 3/1 > > >>> > ums0 at uhidev1: 3 buttons, Z dir > > >>> > wsmouse0 at ums0 mux 0 > > >>> > uhub4 at uhub1 port 1 configuration 1 interface 0 "Intel Rate > Matching > > >>> Hub" > > >>> > rev 2.00/0.05 addr 2 > > >>> > vscsi0 at root > > >>> > scsibus2 at vscsi0: 256 targets > > >>> > softraid0 at root > > >>> > scsibus3 at softraid0: 256 targets > > >>> > root on sd0a (705899aa0befef4d.a) swap on sd0b dump on sd0b > > >>> > > >> > > >> > > >> I have a firewall test setup consisting of 14 machines. > > >> All old E3's, with a mix of ix and em, and some 2x40G ixl(work in > > >> progress). > > >> > > >> Firewalls interfaces are configured as: > > >> 2x ix => trunk(lacp) => vlans+carps > > >> > > >> 14 vlan interfaces and 13 carps (no carp on pfsync vlan interface) > > >> set skip on lo and the vlan carrying pfsync. > > >> No shaping or other funkiness, just pass/block. > > >> > > >> As this is a forwarding performance test with very low session rate > > >> the rule-set is irrelevant. > > >> > > >> The test performed is ping-pong of UDP packets across the firewall > > >> at the speed of RTT, additional clients are added to increase load. > > >> > > >> Testing with OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar 3 10:24:08 > MST > > >> 2019 > > >> Baseline, single session between testers. > > >> OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar 3 10:24:08 MST 2019 > > >> > > >> thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -c 25000 > > >> 10.96.2.21:2222 > > >> packet size: 512 > > >> packets per client: 25000 > > >> number of clients: 1 > > >> server address: 10.96.2.21:2222 > > >> pps: 4016 total drops: 0 avg rtt: 0.249 ms clients: 1 > > >> pps: 4015 total drops: 0 avg rtt: 0.249 ms clients: 1 > > >> pps: 3905 total drops: 0 avg rtt: 0.256 ms clients: 1 > > >> pps: 4016 total drops: 0 avg rtt: 0.249 ms clients: 1 > > >> pps: 4015 total drops: 0 avg rtt: 0.249 ms clients: 1 > > >> pps: 3976 total drops: 0 avg rtt: 0.252 ms clients: 1 > > >> Runtime: 7.118854881s Packets received: 25000 Packets dropped: 0 > > >> thehoff@cloud5.swe69$ > > >> > > >> The pps here is packets sent (and received), so firewall pps > > >> on trunk is double the above number. > > >> > > >> A ramp-up test where one client per second is added until packet loss: > > >> > > >> thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -r 1 > > >> 10.96.2.21:2222 > > >> packet size: 512 > > >> ramp-up interval: 1 seconds > > >> packets per client: 9223372036854775807 > > >> number of clients: 9223372036854775807 > > >> server address: 10.96.2.21:2222 > > >> pps: 4051 total drops: 0 avg rtt: 0.494 ms clients: 2 > > >> pps: 5736 total drops: 0 avg rtt: 0.523 ms clients: 3 > > >> pps: 8258 total drops: 0 avg rtt: 0.484 ms clients: 4 > > >> pps: 10852 total drops: 0 avg rtt: 0.461 ms clients: 5 > > >> pps: 13174 total drops: 0 avg rtt: 0.455 ms clients: 6 > > >> pps: 15417 total drops: 0 avg rtt: 0.454 ms clients: 7 > > >> pps: 17894 total drops: 0 avg rtt: 0.447 ms clients: 8 > > >> pps: 20010 total drops: 0 avg rtt: 0.450 ms clients: 9 > > >> pps: 22178 total drops: 0 avg rtt: 0.451 ms clients: 10 > > >> pps: 24252 total drops: 0 avg rtt: 0.454 ms clients: 11 > > >> pps: 26056 total drops: 0 avg rtt: 0.461 ms clients: 12 > > >> pps: 28060 total drops: 0 avg rtt: 0.463 ms clients: 13 > > >> pps: 30437 total drops: 0 avg rtt: 0.460 ms clients: 14 > > >> pps: 32503 total drops: 0 avg rtt: 0.461 ms clients: 15 > > >> pps: 33421 total drops: 0 avg rtt: 0.479 ms clients: 16 > > >> pps: 35810 total drops: 0 avg rtt: 0.475 ms clients: 17 > > >> pps: 37042 total drops: 0 avg rtt: 0.486 ms clients: 18 > > >> pps: 38626 total drops: 0 avg rtt: 0.492 ms clients: 19 > > >> read failed: read udp 10.96.3.52:43580->10.96.2.21:2222: i/o timeout > > >> pps: 38045 total drops: 1 avg rtt: 0.526 ms clients: 20 > > >> read failed: read udp 10.96.3.52:3459->10.96.2.21:2222: i/o timeout > > >> > > >> Runtime: 20.198108441s Packets received: 472256 Packets dropped: 2 > > >> thehoff@cloud5.swe69$ > > >> > > >> Packet loss starts at around 35-40kpps: > > >> thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -c 25000 -n 17 > > >> 10.96.2.21:2222 > > >> packet size: 512 > > >> packets per client: 25000 > > >> number of clients: 17 > > >> server address: 10.96.2.21:2222 > > >> pps: 35234 total drops: 0 avg rtt: 0.482 ms clients: 17 > > >> pps: 37294 total drops: 0 avg rtt: 0.456 ms clients: 17 > > >> pps: 36400 total drops: 0 avg rtt: 0.467 ms clients: 17 > > >> pps: 37040 total drops: 0 avg rtt: 0.459 ms clients: 17 > > >> pps: 37081 total drops: 0 avg rtt: 0.458 ms clients: 17 > > >> pps: 35258 total drops: 0 avg rtt: 0.482 ms clients: 17 > > >> read failed: read udp 10.96.3.52:45255->10.96.2.21:2222: i/o timeout > > >> pps: 36918 total drops: 1 avg rtt: 0.460 ms clients: 17 > > >> ^C > > >> Runtime: 8.029600463s Packets received: 285855 Packets dropped: 1 > > >> thehoff@cloud5.swe69$ > > >> > > >> > > >> Now testing with OpenBSD 6.5-beta (GENERIC.MP) #760: Thu Feb 28 > 12:38:40 > > >> MST 2019. > > >> Single-session base line: > > >> > > >> thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -c 25000 > > >> 10.96.2.21:2222 > > >> packet size: 512 > > >> packets per client: 25000 > > >> number of clients: 1 > > >> server address: 10.96.2.21:2222 > > >> pps: 3640 total drops: 0 avg rtt: 0.275 ms clients: 1 > > >> pps: 3568 total drops: 0 avg rtt: 0.280 ms clients: 1 > > >> pps: 3638 total drops: 0 avg rtt: 0.275 ms clients: 1 > > >> pps: 3637 total drops: 0 avg rtt: 0.275 ms clients: 1 > > >> pps: 3641 total drops: 0 avg rtt: 0.275 ms clients: 1 > > >> pps: 3674 total drops: 0 avg rtt: 0.272 ms clients: 1 > > >> Runtime: 7.131570891s Packets received: 25000 Packets dropped: 0 > > >> thehoff@cloud5.swe69$ > > >> > > >> Ramp-up test: > > >> > > >> thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -r 1 > > >> 10.96.2.21:2222 > > >> > > >> > > >> > > >> [2/1836] > > >> packet size: 512 > > >> ramp-up interval: 1 seconds > > >> packets per client: 9223372036854775807 > > >> number of clients: 9223372036854775807 > > >> server address: 10.96.2.21:2222 > > >> pps: 3827 total drops: 0 avg rtt: 0.523 ms clients: 2 > > >> pps: 5686 total drops: 0 avg rtt: 0.528 ms clients: 3 > > >> pps: 8278 total drops: 0 avg rtt: 0.483 ms clients: 4 > > >> pps: 10807 total drops: 0 avg rtt: 0.463 ms clients: 5 > > >> pps: 13402 total drops: 0 avg rtt: 0.448 ms clients: 6 > > >> pps: 15878 total drops: 0 avg rtt: 0.441 ms clients: 7 > > >> pps: 18432 total drops: 0 avg rtt: 0.434 ms clients: 8 > > >> pps: 20948 total drops: 0 avg rtt: 0.430 ms clients: 9 > > >> pps: 23393 total drops: 0 avg rtt: 0.427 ms clients: 10 > > >> pps: 25905 total drops: 0 avg rtt: 0.425 ms clients: 11 > > >> pps: 28186 total drops: 0 avg rtt: 0.426 ms clients: 12 > > >> pps: 29716 total drops: 0 avg rtt: 0.437 ms clients: 13 > > >> pps: 31801 total drops: 0 avg rtt: 0.440 ms clients: 14 > > >> pps: 32901 total drops: 0 avg rtt: 0.456 ms clients: 15 > > >> pps: 34493 total drops: 0 avg rtt: 0.464 ms clients: 16 > > >> pps: 35223 total drops: 0 avg rtt: 0.483 ms clients: 17 > > >> pps: 36824 total drops: 0 avg rtt: 0.489 ms clients: 18 > > >> pps: 38655 total drops: 0 avg rtt: 0.492 ms clients: 19 > > >> pps: 39625 total drops: 0 avg rtt: 0.505 ms clients: 20 > > >> pps: 41467 total drops: 0 avg rtt: 0.506 ms clients: 21 > > >> pps: 43342 total drops: 0 avg rtt: 0.508 ms clients: 22 > > >> pps: 44565 total drops: 0 avg rtt: 0.516 ms clients: 23 > > >> pps: 47022 total drops: 0 avg rtt: 0.510 ms clients: 24 > > >> pps: 48825 total drops: 0 avg rtt: 0.512 ms clients: 25 > > >> pps: 49426 total drops: 0 avg rtt: 0.526 ms clients: 26 > > >> pps: 51010 total drops: 0 avg rtt: 0.529 ms clients: 27 > > >> pps: 53227 total drops: 0 avg rtt: 0.526 ms clients: 28 > > >> pps: 55427 total drops: 0 avg rtt: 0.523 ms clients: 29 > > >> pps: 56294 total drops: 0 avg rtt: 0.533 ms clients: 30 > > >> pps: 57202 total drops: 0 avg rtt: 0.542 ms clients: 31 > > >> pps: 58672 total drops: 0 avg rtt: 0.545 ms clients: 32 > > >> pps: 60757 total drops: 0 avg rtt: 0.543 ms clients: 33 > > >> pps: 61619 total drops: 0 avg rtt: 0.552 ms clients: 34 > > >> pps: 62094 total drops: 0 avg rtt: 0.564 ms clients: 35 > > >> pps: 65039 total drops: 0 avg rtt: 0.554 ms clients: 36 > > >> pps: 65980 total drops: 0 avg rtt: 0.561 ms clients: 37 > > >> pps: 68124 total drops: 0 avg rtt: 0.558 ms clients: 38 > > >> pps: 68712 total drops: 0 avg rtt: 0.568 ms clients: 39 > > >> pps: 71362 total drops: 0 avg rtt: 0.561 ms clients: 40 > > >> pps: 72715 total drops: 0 avg rtt: 0.564 ms clients: 41 > > >> pps: 72362 total drops: 0 avg rtt: 0.580 ms clients: 42 > > >> pps: 72946 total drops: 0 avg rtt: 0.589 ms clients: 43 > > >> pps: 77915 total drops: 0 avg rtt: 0.565 ms clients: 44 > > >> pps: 77999 total drops: 0 avg rtt: 0.577 ms clients: 45 > > >> pps: 78544 total drops: 0 avg rtt: 0.586 ms clients: 46 > > >> pps: 80823 total drops: 0 avg rtt: 0.582 ms clients: 47 > > >> pps: 82208 total drops: 0 avg rtt: 0.584 ms clients: 48 > > >> pps: 83035 total drops: 0 avg rtt: 0.590 ms clients: 49 > > >> pps: 84476 total drops: 0 avg rtt: 0.592 ms clients: 50 > > >> pps: 86276 total drops: 0 avg rtt: 0.591 ms clients: 51 > > >> pps: 86409 total drops: 0 avg rtt: 0.602 ms clients: 52 > > >> pps: 87482 total drops: 0 avg rtt: 0.606 ms clients: 53 > > >> pps: 88760 total drops: 0 avg rtt: 0.608 ms clients: 54 > > >> pps: 90796 total drops: 0 avg rtt: 0.606 ms clients: 55 > > >> pps: 92476 total drops: 0 avg rtt: 0.606 ms clients: 56 > > >> pps: 94324 total drops: 0 avg rtt: 0.604 ms clients: 57 > > >> pps: 96194 total drops: 0 avg rtt: 0.603 ms clients: 58 > > >> pps: 98696 total drops: 0 avg rtt: 0.598 ms clients: 59 > > >> pps: 99291 total drops: 0 avg rtt: 0.604 ms clients: 60 > > >> pps: 100693 total drops: 0 avg rtt: 0.606 ms clients: 61 > > >> pps: 103156 total drops: 0 avg rtt: 0.601 ms clients: 62 > > >> pps: 104484 total drops: 0 avg rtt: 0.603 ms clients: 63 > > >> pps: 103819 total drops: 0 avg rtt: 0.616 ms clients: 64 > > >> pps: 105292 total drops: 0 avg rtt: 0.617 ms clients: 65 > > >> pps: 107878 total drops: 0 avg rtt: 0.612 ms clients: 66 > > >> pps: 108857 total drops: 0 avg rtt: 0.615 ms clients: 67 > > >> pps: 108004 total drops: 0 avg rtt: 0.630 ms clients: 68 > > >> pps: 108584 total drops: 0 avg rtt: 0.635 ms clients: 69 > > >> pps: 108634 total drops: 0 avg rtt: 0.644 ms clients: 70 > > >> pps: 109416 total drops: 0 avg rtt: 0.649 ms clients: 71 > > >> pps: 111989 total drops: 0 avg rtt: 0.643 ms clients: 72 > > >> pps: 112612 total drops: 0 avg rtt: 0.648 ms clients: 73 > > >> pps: 113058 total drops: 0 avg rtt: 0.655 ms clients: 74 > > >> pps: 114198 total drops: 0 avg rtt: 0.657 ms clients: 75 > > >> pps: 116640 total drops: 0 avg rtt: 0.652 ms clients: 76 > > >> pps: 117816 total drops: 0 avg rtt: 0.654 ms clients: 77 > > >> pps: 117980 total drops: 0 avg rtt: 0.661 ms clients: 78 > > >> pps: 120047 total drops: 0 avg rtt: 0.658 ms clients: 79 > > >> pps: 123420 total drops: 0 avg rtt: 0.648 ms clients: 80 > > >> pps: 123739 total drops: 0 avg rtt: 0.655 ms clients: 81 > > >> pps: 125930 total drops: 0 avg rtt: 0.651 ms clients: 82 > > >> pps: 127521 total drops: 0 avg rtt: 0.651 ms clients: 83 > > >> pps: 128499 total drops: 0 avg rtt: 0.654 ms clients: 84 > > >> pps: 129330 total drops: 0 avg rtt: 0.657 ms clients: 85 > > >> pps: 131785 total drops: 0 avg rtt: 0.653 ms clients: 86 > > >> pps: 131340 total drops: 0 avg rtt: 0.662 ms clients: 87 > > >> pps: 131660 total drops: 0 avg rtt: 0.668 ms clients: 88 > > >> pps: 133302 total drops: 0 avg rtt: 0.668 ms clients: 89 > > >> pps: 131994 total drops: 0 avg rtt: 0.682 ms clients: 90 > > >> pps: 133166 total drops: 0 avg rtt: 0.683 ms clients: 91 > > >> pps: 132691 total drops: 0 avg rtt: 0.693 ms clients: 92 > > >> pps: 132280 total drops: 0 avg rtt: 0.703 ms clients: 93 > > >> pps: 131950 total drops: 0 avg rtt: 0.712 ms clients: 94 > > >> pps: 133879 total drops: 0 avg rtt: 0.710 ms clients: 95 > > >> pps: 132591 total drops: 0 avg rtt: 0.724 ms clients: 96 > > >> pps: 131854 total drops: 0 avg rtt: 0.736 ms clients: 97 > > >> pps: 132559 total drops: 0 avg rtt: 0.739 ms clients: 98 > > >> pps: 132294 total drops: 0 avg rtt: 0.748 ms clients: 99 > > >> pps: 132859 total drops: 0 avg rtt: 0.753 ms clients: 100 > > >> pps: 131105 total drops: 0 avg rtt: 0.770 ms clients: 101 > > >> ^C > > >> Runtime: 1m41.78992826s Packets received: 8284037 Packets dropped: 0 > > >> thehoff@cloud5.swe69$ > > >> > > >> We are here beyond the point where the tester cpu's can cause false > > >> positives. > > >> > > >> With the older snapshot I can run 4 concurrent 30 client tests without > > >> packet > > >> loss, this is more than 300kpps on trunk0 on the firewall. > > >> Output from one of the four testers: > > >> > > >> thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -c 0 -n 30 > > >> 10.96.2.21:2222 > > >> packet size: 512 > > >> packets per client: 9223372036854775807 > > >> number of clients: 30 > > >> server address: 10.96.2.21:2222 > > >> pps: 36448 total drops: 0 avg rtt: 0.823 ms clients: 30 > > >> pps: 40931 total drops: 0 avg rtt: 0.733 ms clients: 30 > > >> pps: 41419 total drops: 0 avg rtt: 0.724 ms clients: 30 > > >> pps: 41585 total drops: 0 avg rtt: 0.721 ms clients: 30 > > >> pps: 41078 total drops: 0 avg rtt: 0.730 ms clients: 30 > > >> pps: 41637 total drops: 0 avg rtt: 0.721 ms clients: 30 > > >> pps: 41504 total drops: 0 avg rtt: 0.723 ms clients: 30 > > >> pps: 41666 total drops: 0 avg rtt: 0.720 ms clients: 30 > > >> pps: 41803 total drops: 0 avg rtt: 0.718 ms clients: 30 > > >> pps: 42074 total drops: 0 avg rtt: 0.713 ms clients: 30 > > >> pps: 40728 total drops: 0 avg rtt: 0.737 ms clients: 30 > > >> ^C > > >> Runtime: 11.877603832s Packets received: 475599 Packets dropped: 0 > > >> thehoff@cloud5.swe69$ > > >> > > >> The recent snapshot takes me from over 300kpps clean, to less than > 40kpps > > >> clean. > > >> The test clients runs 6.3, as that release is a lot faster here than > 6.4 > > >> or snaps. > > >> > > >> Regards Tony > > >> > > >> > > > On the older snapshot, four tcp sessions with tcpbench resulted in > 350kpps > > > in this setup. > > > The current snapshot gets around 100kpps, fluctuating between 70 and > 120 > > > kpps. > > > > > > Don't be fooled by the named "cloud" on the testers. They are > supermicro > > > microcloud boxes, > > > not cloud-cloud VM's. > > > > > > Regards Tony > > > > > > > > > > > The recent snapshot takes me from over 300kpps > > > clean, to less than 40kpps clean. > > > > My bad, less than 80kpps clean on trunk0. > > Hi Tony, > > This is cos of my ifiq_diff from the end of last week. I'm going to back > it out in the morning when I'm actually awake, probably 12 or so > hours from now. > > Do you want an explanation of why? > > Cheers, > dlg >
Hi David, I think I got the gist of it from your commits, besides I think your time is better spent on the code than on explaining things to me. Regards Tony
