On Friday, Aug 5, 2005, at 15:18 US/Pacific, Karl O. Pinc wrote:
I was also hoping to get some comment from somebody who'd tried
queueing inbound traffic from a WAN link using a 2 port box to see how
successful they were in improving perceived bandwidth.
I have. It was a couple years ago and that configuration is no longer
in use, so most of this is from memory...
The goal was to make Quake 3 playable in the face of web and P2P
traffic. Quake 3 is an online First-Person Shooter game that uses UDP
as a transport, and is sensitive to both latency and dropped packets.
It includes a small onscreen network graph that reflects latency,
jitter, lost packets, and the status of the client side game prediction
engine in an easy-to-read format. It makes a decent network diagnostic
tool, actually...
The servers I played on had code to compensate for latency, so the most
important thing for my setup was getting _consistent_ latency. Lower
latency was preferable but not required. On an otherwise unused 320k
SDSL, latency was around 90ms.
-----
altq on $int_if priq bandwidth 200Kb queue { q3_def, q3_pri }
queue q3_def priority 0 qlimit 1 priq(default)
queue q3_pri priority 7 qlimit 5
pass out on $int_if inet proto udp from any port 27959 >< 27964 to
192.168.0.251 keep state queue q3_pri
pass in on $int_if inet proto udp from 192.168.0.251 to any port 27959
>< 27964 keep state queue q3_pri
-----
IIRC under full load the latency with this configuration was fairly
steady around 200ms, and I could play decent games.
Note that the bandwidth limit is 2/3 line capacity. I always disabled
this queue setup when I was done with the game, as keeping it active
was counterproductive. I didn't spend much time tuning this limit
though, so I don't know how much higher it could have gone and remained
acceptable.
The low queue limits are important. For Quake 3, dropped packets are
preferable to delayed ones; game engine interpolation can recover
drops, but inconsistent latency throws everything off. The limit of 5
is just to keep something in the queue for the priority engine to use.
For the other streams, I wanted them to drop quickly to more closely
reflect a bandwidth limit. Delaying them would give TCP the impression
that the link was simply high-latency, and the result would be bursty
traffic that leads to inconsistent latency for every other stream on
the pipe.
The problem with queue-based prioritization is that in order to have a
choice of things to prioritize, everything must be queued. The mere
act of queueing introduces latency, and for things like TCP, latency
decouples simulated bandwidth from real line capacity.
Stock altq (before the pf merge) contained a simple token bucket
rate-limiter for inbound flows. Were code written to combine it with
pf's state logic, it could be used to force bandwidth of other streams
below a certain limit only when states needing reserve bandwidth were
active. That assumes fixed reserve sizes though. A bonus would be
using smoothed historical flow statistics to predictively limit
bandwidth of all flows dynamically. This would avoid the latency
problem inherent in priority queueing. Implementing such a thing
efficiently would be an interesting project.
I threatened to do some work on the token bucket once, but ran out of
time to work on such a project. Sorry :(
