Oh, this is rich. I read a little bit of one of the links provided:
http://www.mikrotik.com/forum/viewtopic.php?t=9&highlight=nstreme
> Note: We only let our radios go to 24Mbps in order to improve link > stability (might be better now that we are running 2.8.10 (will try > 36Mbps and higher tonight)). > > We are seeing about 15-19Mbps UDP and 10-16Mbps (TCP) across each of > 4 50km+ links running nstreme. This is a BIG improvement per link. > However, when we run test traffic across all 4 router links we only > see 5-6Mpbs TCP. However the UDP numbers are still around 15Mbps. Our > routers are 1GHz processors, 128Mb RAM with 2 or 3 wlan cards in each > so power should not be a problem. I will run some more tests tonight > and pay attention to processor utilization while I do it. > > Any ideas why the routers slow down with TCP when passing traffic > from link to link? > > Thanks, Ken tully Veteran Veteran > > > Joined: 28 May 2004 Posts: 257 > > PostPosted: Wed Jun 02, 2004 9:17 am Post subject: Reply with > quote There is no need to restrict the higher rates of 802.11. There > is an algorithm that calculates which rate will produce the highest > throughput even with lost frames and such. Packets will never be lost > unless the quality at the lowest rate is so bad that the client will > be unregistered. > > One session of TCP will never use the full throughput of a link. Many > multiple sessions of TCP will get close, but a UDP bandwidth test > will give the accurate number. > > John
Wait, they've got 4 radios on each end. They get 15-19Mbps over UDP (which is it?) and 10-16Mbps (TCP) (again, which is it?)
But when they turn *all* the radios on.. presto, speed drop.
Tully blames it on some kind of undocumented lack of efficiency in TCP, but the real problem is... packet loss. TCP reacts rather badly in the face of a lossy link. Turning up the other links means that more packets are smashed, so TCP closes its window.
This is such a well-understood phenomena that it wouldn't even qualify for a senior year research project anymore.
Note that UDP isn't unaffected, the "15-19Mbps" is now "around 15Mbps".
Hello, this is a 25% reduction in throughput. Likely 25% or greater packet loss from "2 to 3 wlan cards". Wait, I thought there were "4 links".
And, btw, packet loss is *assumed* in wireless networks. Ever wonder why rx sensitivity (and lots of other parameters in 802.11) are calculated at some errored frame (or bit error) rate?
The algorithm is "rate fallback", and these are notoriously difficult to get right. The tuning for benchmarks (max throughput) is not quite what you want for the real world (more emphasis on reliability, which tends to translate to an algorithm that is less agressive about seeking the higher modulation rates.)
The other interesting side effect, of course is that as the modulation rate goes down, the capacity of the link goes down. If you've got more than one 'client" on the link (say, a MPTP system), then your slower clients take up more of the "capacity" of the cell.
I won't go into the "capture" issues here.
And then there is this:
We have removed NStreme from all of our routers until the winbox interface is available. We found that even though single links were very fast the routers (routerboards) could not keep up with the traffic. 4 links that were about 10-16Mbs (TCP, 50km) each would only yield 3-5Mbits of TCP through put. I tried many connections and could not get anything higher. I would LOVE to have a 10Mbs total throughput connections along these 4 routers but so far we get better total performance without NStreme.
likely that before the "close by" radios were prevented from transmitting because the RSSI of any one outbound packet was high enough to set CCA in the other radios.
I find it likely that some "programmer" at Microtik set the CCA threshold down, and this combined with the (should I say 'obvious'?) TDMA protocol ("NStreme") yeilds some really hairy co-interference issues.
But we've been over that, and I said I wouldn't talk about it any more.
I am nearly entertained.
Jim
