On Mon, Nov 26, 2018 at 1:30 PM Jonathan Morton <chromati...@gmail.com> wrote: > > > On 26 Nov, 2018, at 9:08 pm, Pete Heist <p...@heistp.net> wrote: > > > > So I just thought to continue the discussion- when does the CoDel part of > > fq_codel actually help in the real world? > > Fundamentally, without Codel the only limits on the congestion window would > be when the sender or receiver hit configured or calculated rwnd and cwnd > limits (the rwnd is visible on the wire and usually chosen to be large enough > to be a non-factor), or when the queue overflows. Large windows require > buffer memory in both sender and receiver, increasing costs on the sender in > particular (who typically has many flows to manage per machine).
You can run devices out of memory more easily with our current ecn implementations. I am seeing folk cut memory per instance to 256k in, for example, the gluon project. we end up dropping (which is better than the device crashing), and in the fq_codel case, *bulk* head dropping. I see a bifurcation on the data when we do this, and I have a one line patch to the fq_codel bulk dropper that appears to make things better when extremely memory constrained like this, but haven't got around to fully evaluating it: https://github.com/dtaht/fq_codel_fast/commit/a524fc2e39dc291199b9b04fb890ea1548f17641 would rather like more to try the memory limits we are seeing in the field. 32MB (fq_codel default) is waaaay too much. 4MB is way too much even for gbit, I think. 256k? well, given the choice between crashing or not... > > Queue overflow tends to result in burst loss and head-of-line blocking in the > receiver, which is visible to the user as a pause and subsequent jump in the > progress of their download, accompanied by a major fluctuation in the > estimated time to completion. The lost packets also consume capacity > upstream of the bottleneck which does not contribute to application > throughput. These effects are independent of whether overflow dropping > occurs at the head or tail of the bottleneck queue, though recovery occurs > more quickly (and fewer packets might be lost) if dropping occurs from the > head of the queue. > > From a pure throughput-efficiency standpoint, Codel allows using ECN for > congestion signalling instead of packet loss, potentially eliminating packet > loss and associated lead-of-line blocking entirely. Even without ECN, the > actual cwnd is kept near the minimum necessary to satisfy the BDP of the > path, reducing memory requirements and significantly shortening the recovery > time of each loss cycle, to the point where the end-user may not notice that > delivery is not perfectly smooth, and implementing accurate completion time > estimators is considerably simplified. I wish we had fractional cwnd below 1 and/or that pacing did not rely on cwnd at all. too many flows at any rate you choose, can end up marking 100% of packets, still not run you out of memory, and cause delay. > > An important use-case is where two sequential bottlenecks exist on the path, > the upstream one being only slightly higher capacity but lacking any queue > management at all. This is presently common in cases where home CPE > implements inbound shaping on a generic ISP last-mile link. In that case, > without Codel running on the second bottleneck, traffic would collect in the > first bottleneck's queue as well, greatly reducing the beneficial effects of > FQ implemented on the second bottleneck. In this topology, the overall > effect is inter-flow as well as intra-flow. > > The combination of Codel with FQ is done in such a way that a separate > instance of Codel is implemented for each flow. This means that congestion > signals are only sent to flows that require them, and non-saturating flows > are unmolested. This makes the combination synergistic, where each component > offers an improvement to the behaviour of the other. also a good bullet point somewhere! > > - Jonathan Morton > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 _______________________________________________ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
