On Feb 7, 2014, at 6:58 PM, Toke Høiland-Jørgensen <[email protected]> wrote:
> Nicolas KUHN <[email protected]> writes: > >> Please let us know what you think of the current document. > > Hi Nicolas and others > > Thank you for the draft; here are some comments, mostly of a more > general nature (as opposed to concrete proposals for rewording). > > > 1. TESTING METHODOLOGY AND REPRODUCIBILITY > I think it is important to encourage good testing methodology in a > document like this; and from the current draft the actual proposed > testing method is not clear. > We tried to be the clearer the possible. If you have any comments or proposals, we would be glad to hear them. > I realise it is also important to leave some room for people to do > things in the way that suits their work-flow, however I believe some > minimum standards for methodology would be appropriate. We mentioned in the draft that the metrics and scenarios can be used for any AQM and context (that is the main idea of the draft). The idea is to provide a metric section that can be exploited whatever the context is. Also, the first part of Evaluation section is dedicated to scenarios that are "context-free". I agree that methodology should be integrated in the draft. I will include a dedicated section in the next version of the draft. > Reproducibility > is a part of this that is especially important, but other areas that > spring to mind are: > > - A sufficiently detailed description of the test setup to allow others > to replicate it. Including software and hardware versions etc. > - Availability of the data used for analysis to allow others to check > the conclusions drawn. > > Requiring a reference implementation to be available could also be > relevant to this general point; I would think it would be a reasonable > requirement to have. > > 2. COMPETING VS LOAD-INDUCING TRAFFIC CHARACTERISTICS > Distinguishing between the effect an AQM has on the traffic that induces > the congestion and traffic that competes with it is worthwhile, in my > opinion. For some applications (e.g. background transfers), maximising > goodput for the "offending" stream is less important than ensuring that > other traffic is not experiencing added latency because of it, while for > others (e.g. an adaptive video conference stream), the latency of the > "offending" stream can be vital. Having this distinction in the > measurements (by measuring both) makes it possible to evaluate this > aspect of the AQM behaviour. > I think this is dealt with in section 3.3.1.5. The next version will be more specific with the expected output (measuring the metrics separately for each class of traffic). The next version will also detail that the TCp transfer can be operated with an LBE congestion control. > 3. SOME COMMENTS ON THE SCENARIOS > I have various comments on the proposed testing scenarios: > > - 10 Mbps is a fairly high bandwidth compared to the average connection > to the internet. I believe it would be worthwhile to include lower > bandwidth(s) in the "general" scenarios, and not just in the "optional" > ones. ACK - next version will consider 1Mbps > - I don't believe the scenario dubbed "Wi-Fi" is a realistic > representation of the behaviour of a wifi link. A comprehensive wifi > test would include the presence of multiple hosts, bandwidth varying > on very short time scales, and packet aggregation behaviour. I would > suggest keeping the existing scenario, but calling it something else, > and then adding a more comprehensive wifi scenario (which would > probably need some hashing out). > - I'm missing measurements of the behaviour of real-world traffic such > as DNS lookups, HTTP requests, etc. Indeed, having bidirectional > traffic included would be good. > The idea behind the draft was to be quite generic. Wi-Fi is a use case for "Medium bandwidth, medium delay" scenario. We could go deeper into details, but I am afraid that the draft will lose some of its generically. This should be further discussed with the list. > 4. SCHEDULING vs AQM > In the first section(s) mention AQM and scheduling as though they are > interchangeable, while a later section mentions AQM interaction with > scheduling mechanisms. I believe it is worthwhile to both (a) > distinguish clearly between what is mean by AQM resp scheduling, but > also (b) make the recommendations apply equally to a "pure" AQM > algorithm and to a combination scheme that includes both an AQM and a > scheduler (which may or may not be intricately linked in their > operation). > I am not sure the current draft mention that AQM and scheduling are interchangeable. I think the draft mentions that they may both reduce the latency and the distinction between the two is clear. If some sections lack of clarity, please point them out. > 5. STABILITY ANALYSIS > While I agree that it is important for an algorithm proposal to argue > plausibly for its stability, I think it is wrong to *require* that > argument to be made in the form of a control theoretic analysis. Such a > requirement limits possible algorithms to those that can be described > using that particular paradigm (loosely, things expressible and solvable > by differential equations). For instance, the fluid dynamic TCP model > commonly used for control theory stability analysis, discards a lot of > the dynamics of TCP, most notably slow start and other transient > behaviour, and focuses on the steady state. Since it is by no means > clear that this is the only way to make the argument (and indeed > focusing too much on the steady state can be counter-productive), I > would suggest to at least reword the section to mention control theory > as *a* way to make the argument, rather than the *only* way. > > The draft claims that the impact of every externally tuned parameter MUST be discussed. The tester SHOULD provide background material showing control theoretic analysis but COULD use other ways to discuss stability. Kind regards, Nicolas > > Sorry if this got a bit long; I started with a much shorter list of > notes, I swear! :) > > -Toke _______________________________________________ aqm mailing list [email protected] https://www.ietf.org/mailman/listinfo/aqm
