This is a review of AQM Evaluation Guidelines (draft-kuhn-aqm-eval-guidelines-02)
I think this document has a useful set of tests. I think the document has value and an updated version of this document should be published. There are likely to be more variants of AQM schemes that emerge and this provides a valuable framework to help compare them and to provide better insight into their properties. I sent a set of minor corrections and formatting issues to the document editors as a separate email - Ill not list them here, since these are now planned for the next revision. Ill focus on three topics: (1) The document is a standards-track specification of tests. While the tests seemed generally good, I found the original recommendations inconsistent. Section 1: I think this section is informative. I propose moving the one new requirement (recommending documenting drop-tail comparison is placed in the relevant section. The language needs to be adjusted to avoid RFC 2119 language until this is defined. Section 2: There is a RFC2119 keyword at the start of this section. I think to may be good to consider how this is expressed: These metrics are RECOMMENDED to assess the performance of an AQM scheme. This could be better perhaps to say: This section provides normative requirements for metrics that can be used to assess the performance of an AQM scheme. - I suggest rephrasing one sentence to: It is therefore not necessary to measure all of the following metrics, and guidance is provided for each metric. This is to avoid the possibility that the reader thinks all recommendations are simply optional! - Section 2.3: Packet loss synchronization - Does not make a recommendation, is this SHOULD, MUST or MAY - saying it is important is not sufficient. I suspect it is a MUST. - I think sections 3.1 and 3.2 are informative. Section 4.4 - likewise. In general, I think the RFC2119 usage needs to be tightened to clearly conclude what is to be tested in each case, whether the test is optional, etc. Specifically, I think the language for the requirement should be to evaluate each of the test cases, not just to implement a scenario. - Section 12: This combines SHOULD with sufficiently detailed - is it ok to say SHOULD provide a detailed or something that does leaves the judgment of sufficient part to the evaluater, ensuring some detail is provided. - It may be useful to add a summary table. I do like the idea of a summary table at the end, but to be clear if this is done, please use the EXACT RFC2119 keywords from the sections, to prevent people detecting inconsistencies here. (2) Scenarios and test cases - I think we need to be clear (above) what is required. Im also suggest to introduce a multi-AQM scenario where we identify that there is a need to consider the implications of more than one congested bottleneck along a path that has an active AQM. This case came out of Gen-ART review for the AQM Recommendations, and I think we should identify this as a test case: e.g. "Transports operating under the control of AQM experience the effect of multiple control loops that react over different timescales. It is therefore important that proposed AQM schemes are seen to be stable when they are deployed at multiple points of potential congestion along an Internet path. The pattern of congestion signals (loss or ECN-marking) arising from AQM methods also need to not adversely interact with the dynamics of the transport protocols that they control. (3) I have sent text (offlist) that attempts to correct some deviations in language usage from the AQM recommendations ID, and suggested to cite this where possible, rather than say things similar. These details proved to be minor, I suggested some additional points and rewording for others. I think though a small amount of additional work is still needed to agree which test evaluations and discussions are a MUST and which are a MAY and there seems to be some contradictions, which willneed to be checked in the next revision. Best wishes, Gorry _______________________________________________ aqm mailing list aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm
