Nicolas KUHN <[email protected]> writes: > I believe the draft must be the most generic the possible. These > guidelines provide the tools and aspects that must be looked at, > however the AQM is tested. Even-driven simulations (such as in > NS-2,OMNET) enable to achieve an economical and fast protocol > experimentation [http://arxiv.org/pdf/1002.2827.pdf]. These guidelines > cannot reject a proposal because it has not been tested in the 'real' > world.
Sure, I'm not saying simulation is not useful. I'm just saying it should not be the only evaluation of an algorithm. > I will add in the "methodology" section something about that issue, > saying that proposals SHOULD be evaluated in testbeds, and MAY be > evaluated with event-driven simulations to better understand their > internal behaviour. Great. Would add "reference implementation available" to that. :) > This is the kind of discussion that should be avoided in the draft. > Indeed, I am afraid we may open a "pandora's box" as we do not want to > list all the issues encountered by any evaluation tool set. Not all of them perhaps, but the most common, and those that apply generally across tool sets. > We may end discussing why choosing emulation, vs simulation, vs > mathematical model, vs real-workd experiment. A discussion that is sorely lacking, IMO :) >> ...case where latency is measured by a separate flow (such as a straight >> 'ping' alongside the test) vs directly measuring the queueing delay >> of the packets making up the traffic scenario; > > I think that this is more a metric linked to the evaluation tool set. > If we want the guidelines to be "generic" and "evaluation tool set" > free, this should not be discussed. My interest here was more to avoid comparing apples to oranges: the two different kinds of measurements can give widely varying results. As an example, a strict priority queue would give very good latency for flows alongside the main "elephant", but could completely starve the elephant itself. Which may be desirable in some cases: but distinguishing between different types of behaviour is important if we want to compare different algorithms fairly. And I think it is worth specifying that in the draft somewhere. > I am not sure that it makes sense here, that would be another > pandora's box: the guidelines should not consider every possible TCP > improvements. Otherwise, others would say "do you consider RENO? > CUBIC? Delay based CC?" or "what about MultiPath?". We must generic > and speak generally about TCP and UDP - without much more details. Well, the draft does mention CUBIC under "aggressive" senders. But if you don't want to enumerate all the permutations, some generic language (along the lines of "SHOULD be evaluated against the most commonly seen TCP variants deployed on the internet (for example, CUBIC, IW4, IW10, etc"). > Yes I updated the Section "Medium bandwidth, medium delay: Wi-Fi" so > that the tester may include "small time scales varying bandwidth" Right, better. Still not entirely sure that it is a good representation of wifi, but I'm not going to argue the point vehemently... :) > I agree as well - the draft would then need some rewording > > I will add an entry in the glossary explaining that by AQM, the draft > consider hybrid AQM+scheduling > > I will update the Section (4.4) considering that scheduling is a > feature of an AQM, and is not introduced "on top" of it. Great! >> seems a bit speculative to me. Has the fact that the algorithms' >> activation time is a predictor of adverse side effects actually been >> demonstrated? > > The guidelines just mention that this MAY be destructive and this > issue should be carefully discussed. Yeah, I was just puzzled by why specifically one configuration of activation time interaction is mentioned as potentially problematic, instead of just saying "potentially destructive interactions between AQM and scheduling should be accounted for" or somesuch. -Toke
signature.asc
Description: PGP signature
_______________________________________________ aqm mailing list [email protected] https://www.ietf.org/mailman/listinfo/aqm
