First about the setup: a bit older hardware - P2B mobo, P2 333, 192 mb
ram, compex pci nics (dmesg attached at the end). 2 boxes - 1 obsd with
old 20gb disk, and one ftp client on w2k with even older 3gb hdd.
OBSD box is pretty much vanilla 3.8 with vsftpd accepting passive
connections at 40000-40050. The other w2k machine acts as a ftp client.
Used pf setup (*very simplified* to only show the issue) is as follows
(75 is windows machine):
#-----------
test="192.168.100.75"
int_if="wb0"
set skip on lo0
scrub in on $int_if
block drop all
pass in on $int_if inet from $test to any keep state
pass out on $int_if inet from any to any keep state
pass in on $int_if inet proto tcp from $test to any port \
{21,40000:40050} keep state
pass in on $int_if inet proto tcp from $test to any port \
{22} keep state
#-----------
In this scenarion (or with pf disabled), everything works like a charm.
I can get throughput pretty much limited by what w2k and its antique hdd
can handle (around 16 Mb - 20 Mb).
Similar setup with priq queues works also without any problems:
#-----------
test="192.168.100.75"
int_if="wb0"
set skip on lo0
scrub in on $int_if
altq on $int_if bandwidth priq queue {std, ftp, pri, ack}
queue std on $int_if priority 1 priq(default)
queue ftp on $int_if priority 2 priq
queue pri on $int_if priority 3 priq
queue ack on $int_if priority 4 priq
block drop all
pass in on $int_if inet from $test to any keep state queue (std, ack)
pass out on $int_if inet from any to any keep state queue (std, ack)
pass in on $int_if inet proto tcp from $test to any port \
{21,40000:40050} keep state queue (ftp, ack)
pass in on $int_if inet proto tcp from $test to any port \
{22} keep state queue (pri, ack)
#-----------
But....
If I change altq line and set bandwidth to something smaller - like 10Mb
- problems show up. Throughput on ftp drops brutally to around 150 - 250 Kb
Also if I use for example cbq in the following way (regardless if
bandwidth is or isn't explicitely set, and to what value):
altq on $int_if bandwidth <whatever> cbq queue {std, ftp, pri, ack}
queue std bandwidth 2Mb priority 1 cbq(default)
queue ftp bandwidth 4Mb priority 2 cbq
queue pri bandwidth 2Mb priority 3 cbq
queue ack bandwidth 2Mb priority 4 cbq
Transfers on ftp are around the same - 150 - 250 Kb, instead of what
they should be - around 4 Mb
I did some experiments - for example setting parent queue to 100mbit and
allowing childs to borrow, will fix the problem. Or setting parent to
100mbit and childs to 20/40/20/20 with or without borrowing. But for
example, parent at 100, and ftp child set to 10 or 15 won't do much
good, besides small increase. "Good" starts when I set the bw to around
20 Mb on ftp child (or when it can borrow this amount), although at that
value, the transfer was "choking" between 500Kb and max (more often low
than high), with average around 4.8Mb on 30 megabyte file.
Tests with other tools - sftp, scp - (where I can get around 10 Mb
normally) have similar stalls, if assigned bw is set to anything
"smaller".
I'm out of ideas what could be wrong. Above setups are trival and don't
really differ from examples that can be found in man or obsd faq, or
anywhere else for that matter. There were no extra cpu/memory load or
problems after enabling queing, or any unusual things reported by pfctl
in verbose mode.
Of course, upload to bsd machine always worked fine (no queueing applied
to incoming packets).
Any idea what could be wrong ? Some strange incompatibility with
[network] driver (maybe compex NICs are problematic for some reason -
I know they aren't precisely "state of the art" of what a nic could be) ?
Maybe some simple mistake on my side I'm not aware of ?
Attached dmesg:
OpenBSD 3.8 (GENERIC) #138: Sat Sep 10 15:41:37 MDT 2005
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel Pentium II ("GenuineIntel" 686-class, 512KB L2 cache) 334 MHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR
real mem = 200843264 (196136K)
avail mem = 176390144 (172256K)
using 2477 buffers containing 10145792 bytes (9908K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(bd) BIOS, date 08/27/98, BIOS32 rev. 0 @ 0xfadf0
apm0 at bios0: Power Management spec V1.2
apm0: AC on, battery charge unknown
apm0: flags 70102 dobusy 1 doidle 1
pcibios0 at bios0: rev 2.1 @ 0xf0000/0xb268
pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfd8f0/144 (7 entries)
pcibios0: PCI Exclusive IRQs: 10 11
pcibios0: PCI Interrupt Router at 000:07:0 ("Intel 82371SB ISA" rev 0x00)
pcibios0: PCI bus #1 is the last bus
bios0: ROM list: 0xc0000/0x8000
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "Intel 82443BX AGP" rev 0x02
ppb0 at pci0 dev 1 function 0 "Intel 82443BX AGP" rev 0x02
pci1 at ppb0 bus 1
pcib0 at pci0 dev 7 function 0 "Intel 82371AB PIIX4 ISA" rev 0x02
pciide0 at pci0 dev 7 function 1 "Intel 82371AB IDE" rev 0x01: DMA, channel 0
wired to compatibility, channel 1 wired to compatibility
wd0 at pciide0 channel 0 drive 0: <FUJITSU MPF3204AT>
wd0: 16-sector PIO, LBA, 19546MB, 40031712 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
atapiscsi0 at pciide0 channel 1 drive 1
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: <TOSHIBA, CD-ROM XM-6602B, 1017> SCSI0 5/cdrom
removable
cd0(pciide0:1:1): using PIO mode 4, Ultra-DMA mode 2
uhci0 at pci0 dev 7 function 2 "Intel 82371AB USB" rev 0x01pci_intr_map: no
mapping for pin D
: couldn't map interrupt
"Intel 82371AB Power" rev 0x02 at pci0 dev 7 function 3 not configured
vga1 at pci0 dev 8 function 0 "S3 86C968-0" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
wb0 at pci0 dev 10 function 0 "Compex RL100-ATX 10/100" rev 0x00: irq 10
address 00:80:48:dd:b7:89
ukphy0 at wb0 phy 0: Generic IEEE 802.3u media interface
ukphy0: OUI 0x00601d, model 0x0033, rev. 1
ukphy1 at wb0 phy 1: Generic IEEE 802.3u media interface
ukphy1: OUI 0x00601d, model 0x0033, rev. 1
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
sysbeep0 at pcppi0
npx0 at isa0 port 0xf0/16: using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
biomask fbfd netmask fffd ttymask ffff
pctr: 686-class user-level performance counters enabled
mtrr: Pentium Pro MTRR support
dkcsum: wd0 matches BIOS drive 0x80
root on wd0a
rootdev=0x0 rrootdev=0x300 rawdev=0x302
