Hi Raimundo, please use max directive:
queue root on alc0 bandwidth 600M, max 500M -- Best regards, Loïc BLOT, Engineering UNIX Systems, Security and Network Engineer http://www.unix-experience.fr Le mardi 12 août 2014 à 02:11 -0300, Raimundo Santos a écrit : > Hello misc! > > I am with a very non expected behaviour. With this simple pf.conf > > # pfctl -vnf /etc/pf.conf > > set skip on { lo } > > queue root on alc0 bandwidth 600M default > > pass out on alc0 all flags S/SA set ( queue root ) > > I got this queue output when running tcpbench in client mode > > # pfctl -vvvsq > > [ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 > ] > > [ qlength: 0/ 50 ] > > queue root on alc0 bandwidth 600M default qlimit 50 > > [ pkts: 6099167 bytes: 9233990662 dropped pkts: 0 bytes: 0 > ] > > [ qlength: 0/ 50 ] > > > [ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 > ] > > [ qlength: 0/ 50 ] > > [ measured: 0.0 packets/s, 0 b/s ] > > queue root on alc0 bandwidth 600M default qlimit 50 > > [ pkts: 6500911 bytes: 9842225822 dropped pkts: 0 bytes: 0 > ] > > [ qlength: 0/ 50 ] > > [ measured: 80348.8 packets/s, 973.18Mb/s ] > > > [ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 > ] > > [ qlength: 0/ 50 ] > > [ measured: 0.0 packets/s, 0 b/s ] > > queue root on alc0 bandwidth 600M default qlimit 50 > > [ pkts: 6902593 bytes: 10450369962 dropped pkts: 0 bytes: > 0 ] > > [ qlength: 0/ 50 ] > > [ measured: 80342.6 packets/s, 973.10Mb/s ] > > # pfctl -vsr > > pass out on alc0 all flags S/SA set ( queue root ) > > [ Evaluations: 493 Packets: 14082601 Bytes: 13949048492 States: 1 > ] > > [ Inserted: uid 0 pid 3493 State Creations: 1 ] > > > I've tried with 100M, 200M and 400M, all not shaping. > > I've also tried to setup a root queue with 200M and two child: a default > with 1M and the other, referred in the rule, with 100M, also not working. > > I am playing with tcpbench and this is the only traffic I really care about > on this machine. I restarted the tcpbench client on this machine every time > I reloaded the testing rule and queue, and even deleted the related states > (or states, in cases that I run tcpbench -b <some alias>), but nothing > leads me to the desired bandwidth shaping. > > I am experiencing the same behaviour in a virtual machine under KVM with > PCI Passthrough of an Intel NIC. These are the conf and results from the > virtual machine: > > # pfctl -vf /etc/pf.conf > > > > set skip on { lo } > > queue std on em0 bandwidth 100M default > > pass out on em0 all flags S/SA set ( queue std ) > > > # pfctl -vvvsq > > [ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 > ] > > [ qlength: 0/ 50 ] > > queue std on em0 bandwidth 100M default qlimit 50 > > [ pkts: 1195513815 bytes: 87858084628 dropped pkts: 0 bytes: > 0 ] > > [ qlength: 0/ 50 ] > > > [ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 > ] > > [ qlength: 0/ 50 ] > > [ measured: 0.0 packets/s, 0 b/s ] > > queue std on em0 bandwidth 100M default qlimit 50 > > [ pkts: 1195734870 bytes: 88192747866 dropped pkts: 0 bytes: > 0 ] > > [ qlength: 0/ 50 ] > > [ measured: 44211.0 packets/s, 535.46Mb/s ] > > > [ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 > ] > > [ qlength: 0/ 50 ] > > [ measured: 0.0 packets/s, 0 b/s ] > > queue std on em0 bandwidth 100M default qlimit 50 > > [ pkts: 1195960995 bytes: 88535089028 dropped pkts: 0 bytes: > 0 ] > > [ qlength: 0/ 50 ] > > [ measured: 44718.0 packets/s, 541.60Mb/s ] > > # pfctl -vsr > > pass out on em0 all flags S/SA set ( queue std ) > > [ Evaluations: 2 Packets: 1853414 Bytes: 1708817040 States: 2 > ] > > [ Inserted: uid 0 pid 19622 State Creations: 2 ] > > The traffic passes through a Linux box where I have per ip bandwitdh > control (justifying tcpbench -b <alias>), an in house bandwidth controller > (poor man's 'net equalizer'). My intent was to not put a very high load > over this machine by getting close to my real pps and bps and so make my > capacity planing. > > What am I doing wrong with these queues? > > Thank you all, > Raimundo Santos > > Here is my dmesgs, first from the physical machine and after from the > virtual machine: > > OpenBSD 5.5 (GENERIC.MP) #315: Wed Mar 5 09:37:46 MST 2014 > [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP > real mem = 8538095616 (8142MB) > avail mem = 8302202880 (7917MB) > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 2.5 @ 0x9f400 (53 entries) > bios0: vendor American Megatrends Inc. version "0803" date 07/23/2012 > bios0: ASUSTeK Computer INC. M4A78LT-M-LE > acpi0 at bios0: rev 2 > acpi0: sleep states S0 S1 S3 S4 S5 > acpi0: tables DSDT FACP APIC MCFG OEMB SRAT HPET SSDT > acpi0: wakeup devices PCE2(S4) PCE3(S4) PCE4(S4) PCE5(S4) PCE6(S4) ALAN(S4) > PCE7(S4) PCE9(S4) PCEA(S4) PCEB(S4) PCEC(S4) SBAZ(S4) PS2M(S4) PS2K(S4) > UAR1(S4) P0PC(S4) [...] > acpitimer0 at acpi0: 3579545 Hz, 32 bits > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat > cpu0 at mainbus0: apid 0 (boot processor) > cpu0: AMD Phenom(tm) II X2 565 Processor, 3415.72 MHz > cpu0: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC > cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB > 64b/line 16-way L2 cache, 6MB 64b/line 48-way L3 cache > cpu0: ITLB 32 4KB entries fully associative, 16 4MB entries fully > associative > cpu0: DTLB 48 4KB entries fully associative, 48 4MB entries fully > associative > cpu0: AMD erratum 721 detected and fixed > cpu0: smt 0, core 0, package 0 > mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges > cpu0: apic clock running at 200MHz > cpu0: mwait min=64, max=64, C-substates=0.0.0.0.0, IBE > cpu1 at mainbus0: apid 1 (application processor) > cpu1: AMD Phenom(tm) II X2 565 Processor, 3415.24 MHz > cpu1: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC > cpu1: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB > 64b/line 16-way L2 cache, 6MB 64b/line 48-way L3 cache > cpu1: ITLB 32 4KB entries fully associative, 16 4MB entries fully > associative > cpu1: DTLB 48 4KB entries fully associative, 48 4MB entries fully > associative > cpu1: AMD erratum 721 detected and fixed > cpu1: smt 0, core 1, package 0 > ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 21, 24 pins > acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255 > acpihpet0 at acpi0: 14318180 Hz > acpiprt0 at acpi0: bus 0 (PCI0) > acpiprt1 at acpi0: bus 1 (P0P1) > acpiprt2 at acpi0: bus -1 (PCE2) > acpiprt3 at acpi0: bus -1 (PCE3) > acpiprt4 at acpi0: bus 2 (PCE4) > acpiprt5 at acpi0: bus 3 (PCE6) > acpiprt6 at acpi0: bus 4 (P0PC) > acpicpu0 at acpi0: PSS > acpicpu1 at acpi0: PSS > aibs0 at acpi0 RTMP RVLT RFAN GGRP GITM SITM > acpibtn0 at acpi0: PWRB > cpu0: 3415 MHz: speeds: 3400 2700 2300 800 MHz > pci0 at mainbus0 bus 0 > pchb0 at pci0 dev 0 function 0 "AMD RS780 Host" rev 0x00 > ppb0 at pci0 dev 1 function 0 vendor "Asustek", unknown product 0x9602 rev > 0x00 > pci1 at ppb0 bus 1 > radeondrm0 at pci1 dev 5 function 0 "ATI Radeon HD 3000" rev 0x00 > drm0 at radeondrm0 > radeondrm0: apic 2 int 18 > ppb1 at pci0 dev 4 function 0 "AMD RS780 PCIE" rev 0x00: msi > pci2 at ppb1 bus 2 > re0 at pci2 dev 0 function 0 "Realtek 8168" rev 0x03: RTL8168D/8111D > (0x2800), msi, address 00:e0:4c:68:26:38 > rgephy0 at re0 phy 7: RTL8169S/8110S PHY, rev. 2 > ppb2 at pci0 dev 6 function 0 "AMD RS780 PCIE" rev 0x00: msi > pci3 at ppb2 bus 3 > alc0 at pci3 dev 0 function 0 "Attansic Technology L1C" rev 0xc0: msi, > address f4:6d:04:00:8f:a1 > atphy0 at alc0 phy 0: F1 10/100/1000 PHY, rev. 11 > ahci0 at pci0 dev 17 function 0 "ATI SBx00 SATA" rev 0x00: apic 2 int 22, > AHCI 1.1 > scsibus0 at ahci0: 32 targets > sd0 at scsibus0 targ 0 lun 0: <ATA, ST3500418AS, CC34> SCSI3 0/direct fixed > naa.5000c500120cd875 > sd0: 476940MB, 512 bytes/sector, 976773168 sectors > ohci0 at pci0 dev 18 function 0 "ATI SB700 USB" rev 0x00: apic 2 int 16, > version 1.0, legacy support > ohci1 at pci0 dev 18 function 1 "ATI SB700 USB" rev 0x00: apic 2 int 16, > version 1.0, legacy support > ehci0 at pci0 dev 18 function 2 "ATI SB700 USB2" rev 0x00: apic 2 int 17 > usb0 at ehci0: USB revision 2.0 > uhub0 at usb0 "ATI EHCI root hub" rev 2.00/1.00 addr 1 > ohci2 at pci0 dev 19 function 0 "ATI SB700 USB" rev 0x00: apic 2 int 18, > version 1.0, legacy support > ohci3 at pci0 dev 19 function 1 "ATI SB700 USB" rev 0x00: apic 2 int 18, > version 1.0, legacy support > ehci1 at pci0 dev 19 function 2 "ATI SB700 USB2" rev 0x00: apic 2 int 19 > usb1 at ehci1: USB revision 2.0 > uhub1 at usb1 "ATI EHCI root hub" rev 2.00/1.00 addr 1 > piixpm0 at pci0 dev 20 function 0 "ATI SBx00 SMBus" rev 0x3c: SMI > iic0 at piixpm0 > iic0: addr 0x20 01=11 02=28 04=00 05=00 06=00 07=00 09=00 0a=0f 0b=0f 0c=0f > 0d=0f 0e=19 0f=8a 10=00 11=00 12=00 13=00 14=00 15=0e 16=3a 17=9c 18=80 > 19=fb 1a=af 1b=a0 1c=b6 1d=80 1e=0c 1f=01 20=09 21=09 22=09 23=09 24=03 > 3e=bf words 00=ff11 01=1128 02=28ff 03=ff00 04=0000 05=0000 06=0000 07=00ff > spdmem0 at iic0 addr 0x50: 4GB DDR3 SDRAM PC3-10600 > spdmem1 at iic0 addr 0x51: 4GB DDR3 SDRAM PC3-10600 > pciide0 at pci0 dev 20 function 1 "ATI SB700 IDE" rev 0x00: DMA, channel 0 > configured to compatibility, channel 1 configured to compatibility > pcib0 at pci0 dev 20 function 3 "ATI SB700 ISA" rev 0x00 > ppb3 at pci0 dev 20 function 4 "ATI SB600 PCI" rev 0x00 > pci4 at ppb3 bus 4 > ohci4 at pci0 dev 20 function 5 "ATI SB700 USB" rev 0x00: apic 2 int 18, > version 1.0, legacy support > pchb1 at pci0 dev 24 function 0 "AMD AMD64 10h HyperTransport" rev 0x00 > pchb2 at pci0 dev 24 function 1 "AMD AMD64 10h Address Map" rev 0x00 > pchb3 at pci0 dev 24 function 2 "AMD AMD64 10h DRAM Cfg" rev 0x00 > km0 at pci0 dev 24 function 3 "AMD AMD64 10h Misc Cfg" rev 0x00 > pchb4 at pci0 dev 24 function 4 "AMD AMD64 10h Link Cfg" rev 0x00 > usb2 at ohci0: USB revision 1.0 > uhub2 at usb2 "ATI OHCI root hub" rev 1.00/1.00 addr 1 > usb3 at ohci1: USB revision 1.0 > uhub3 at usb3 "ATI OHCI root hub" rev 1.00/1.00 addr 1 > usb4 at ohci2: USB revision 1.0 > uhub4 at usb4 "ATI OHCI root hub" rev 1.00/1.00 addr 1 > usb5 at ohci3: USB revision 1.0 > uhub5 at usb5 "ATI OHCI root hub" rev 1.00/1.00 addr 1 > isa0 at pcib0 > isadma0 at isa0 > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo > pckbc0 at isa0 port 0x60/5 > pckbd0 at pckbc0 (kbd slot) > pckbc0: using irq 1 for kbd slot > wskbd0 at pckbd0: console keyboard > pcppi0 at isa0 port 0x61 > spkr0 at pcppi0 > lpt0 at isa0 port 0x378/4 irq 7 > it0 at isa0 port 0x2e/2: IT8712F rev 8, EC port 0x290 > usb6 at ohci4: USB revision 1.0 > uhub6 at usb6 "ATI OHCI root hub" rev 1.00/1.00 addr 1 > vscsi0 at root > scsibus1 at vscsi0: 256 targets > softraid0 at root > scsibus2 at softraid0: 256 targets > root on sd0a (5a32fb5b12fd1447.a) swap on sd0b dump on sd0b > drm: initializing kernel modesetting (RS780 0x1002:0x9616 0x1043:0x8388). > radeondrm0: VRAM: 32M 0x00000000C0000000 - 0x00000000C1FFFFFF (32M used) > radeondrm0: GTT: 512M 0x00000000A0000000 - 0x00000000BFFFFFFF > error: [drm:pid0:r600_init_microcode] *ERROR* r600_cp: Failed to load > firmware "radeon-rs780_pfp" > error: [drm:pid0:r600_startup] *ERROR* Failed to load firmware! > error: [drm:pid0:r600_init] *ERROR* disabling GPU acceleration > error: [drm:pid0:radeon_bo_unpin] *ERROR* 0xfffffe821d3ae2b0 unpin not > necessary > error: [drm:pid0:radeon_bo_unpin] *ERROR* 0xfffffe821d3ae2b0 unpin not > necessary > radeondrm0: 1920x1080 > wsdisplay0 at radeondrm0 mux 1: console (std, vt100 emulation), using wskbd0 > wsdisplay0: screen 1-5 added (std, vt100 emulation) > > Virtual machine: > > OpenBSD 5.5 (GENERIC) #271: Wed Mar 5 09:31:16 MST 2014 > [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC > real mem = 1056956416 (1007MB) > avail mem = 1020309504 (973MB) > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xfd9a0 (10 entries) > bios0: vendor Bochs version "Bochs" date 01/01/2007 > bios0: Bochs Bochs > acpi0 at bios0: rev 0 > acpi0: sleep states S3 S4 S5 > acpi0: tables DSDT FACP SSDT APIC HPET SSDT > acpi0: wakeup devices > acpitimer0 at acpi0: 3579545 Hz, 24 bits > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat > acpihpet0 at acpi0: 100000000 Hz > acpiprt0 at acpi0: bus 0 (PCI0) > acpicpu0 at acpi0 > mpbios at bios0 not configured > cpu0 at mainbus0: (uniprocessor) > cpu0: QEMU Virtual CPU version 1.1.2, 3003.29 MHz > cpu0: > FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,CX16,POPCNT,NXE,LONG,LAHF > cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB > 64b/line 16-way L2 cache > cpu0: ITLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped > cpu0: DTLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped > cpu0: smt 0, core 0, package 0 > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges > pci0 at mainbus0 bus 0 > pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02 > pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00 > pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel > 0 wired to compatibility, channel 1 wired to compatibility > pciide0: channel 0 disabled (no drives) > pciide0: channel 1 disabled (no drives) > uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: irq 11 > piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: irq 10 > iic0 at piixpm0 > iic0: addr 0x4c 48=00 words 00=0000 01=0000 02=0000 03=0000 04=0000 05=0000 > 06=0000 07=0000 > iic0: addr 0x4e 48=00 words 00=0000 01=0000 02=0000 03=0000 04=0000 05=0000 > 06=0000 07=0000 > vga1 at pci0 dev 2 function 0 unknown vendor 0x1234 product 0x1111 rev 0x00 > wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) > wsdisplay0: screen 1-5 added (80x25, vt100 emulation) > virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00: Virtio > Network Device > vio0 at virtio0: RingEventIdx disabled by UKC: address 52:54:00:7a:d5:c2 > virtio0: irq 11 > virtio1 at pci0 dev 4 function 0 "Qumranet Virtio Storage" rev 0x00: Virtio > Block Device > vioblk0 at virtio1 > scsibus0 at vioblk0: 2 targets > sd0 at scsibus0 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed > sd0: 1024MB, 512 bytes/sector, 2097152 sectors > virtio1: irq 11 > em0 at pci0 dev 5 function 0 "Intel 82576" rev 0x01: irq 10, address > 00:1b:21:53:51:38 > em1 at pci0 dev 6 function 0 "Intel 82576" rev 0x01: irq 11, address > 00:1b:21:53:51:39 > virtio2 at pci0 dev 7 function 0 "Qumranet Virtio Memory" rev 0x00: Virtio > Memory Balloon Device > viomb0 at virtio2 > virtio2: irq 11 > virtio3 at pci0 dev 8 function 0 "Qumranet Virtio Network" rev 0x00: Virtio > Network Device > vio1 at virtio3: RingEventIdx disabled by UKC: address 52:54:00:57:b8:fa > virtio3: irq 11 > virtio4 at pci0 dev 9 function 0 "Qumranet Virtio Network" rev 0x00: Virtio > Network Device > vio2 at virtio4: RingEventIdx disabled by UKC: address 52:54:00:d0:35:7a > virtio4: irq 10 > isa0 at pcib0 > isadma0 at isa0 > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo > pckbc0 at isa0 port 0x60/5 > pckbd0 at pckbc0 (kbd slot) > pckbc0: using irq 1 for kbd slot > wskbd0 at pckbd0: console keyboard, using wsdisplay0 > pms0 at pckbc0 (aux slot) > pckbc0: using irq 12 for aux slot > wsmouse0 at pms0 mux 0 > pcppi0 at isa0 port 0x61 > spkr0 at pcppi0 > fdc0 at isa0 port 0x3f0/6 irq 6 drq 2 > fd0 at fdc0 drive 0: density unknown > fd1 at fdc0 drive 1: density unknown > usb0 at uhci0: USB revision 1.0 > uhub0 at usb0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 > nvram: invalid checksum > vscsi0 at root > scsibus1 at vscsi0: 256 targets > softraid0 at root > scsibus2 at softraid0: 256 targets > root on sd0a (f03547feaad421a4.a) swap on sd0b dump on sd0b > clock: unknown CMOS layout
