Dear Nicolas,
Thank you very much for the explanation, I have however two questions
regarding your comments:
1) why is the explanation about why are parameters unspecified not
present in the document?
2) in this case, I assume that the remark about "asymmetric link
scenario evaluates an AQM mechanism in a more realistic setup;" in
section 3.1 and all cases where the parameters for capacity and
round-trip time ARE present (see items 2 and 4 below), should be removed ?
I guess now I have to write a longer review... :(
Regards,
Polina
On 12/11/2015 03:20 PM, Kuhn Nicolas wrote:
Dear Polina, all,
Thank you for your email and I understand your concerns. I will
provide more detail answer below for each of your points, but I would
like first to clarify a general issue.
The scope of this document is to present how to assess the performance
of AQMs – without focusing on a specific context (3G networks,
data-centers, satellite networks, home routers, ISP core network)
where Bufferbloat may (or may not) occur (yet). This is the main
reason why this draft does not provide specific details on the
bottleneck capacities, numbers of flows, or traffic generation. Every
strict number or specific flow characteristic limits the scope of this
document that is not only focusing on web traffic latency reduction in
home routers.
Please see inline for specific answer to your concerns.
Cheers,
Nicolas
*De :*aqm [mailto:aqm-boun...@ietf.org] *De la part de* Polina Goltsman
*Envoyé :* lundi 7 décembre 2015 13:32
*À :* Wesley Eddy; aqm@ietf.org
*Cc :* Bless, Roland (TM)
*Objet :* Re: [aqm] I-D Action: draft-ietf-aqm-eval-guidelines-09.txt
Dear all,
Apologies for the long delay.
I have review the latest version of the draft. In my opinion the
document still needs a detailed review about the representation (the
text): the structure of the document would make more sense if the
information in section 2, 3, and 4 would appear in different order,
the document still could use a more consistent terminology, and the
structure of section 5-12 could be more consistent. I am happy to
provide the detailed review, but since it is a lot of work, I have
several concerns about the content of the document, which I would
prefer to clear first.
NK : we changed the ToC of the draft several times and I think the
first sections (2, 3 and 4) are quite independent. I do not see a
specific need for changing their order.
Regarding sections 5-12, we could still add some “Motivation” and
“Recommended tests” for the section 5 for increasing the consistency.
The sections that do not need performance evaluations have another
organization (Motivation then Recommended discussion), but that seems
consistent to me. We had several checks within the list for the
terminology (such as the one done by Gorry), but if you spot anything,
feel free to send your comments.
The document proposes the evaluation methodology for AQMs which
consists of the following topology and scenarios:
0. the experiments should be performed on the topology, described in
Figure 1 in Section 3, where the bottleneck between routers L and R is
of *unspecified* capacity, which SHOULD include*both symmetric and
asymmetric*, and the buffer MAY be set to a BDP. If I understand
correctly, it is further RECOMMENDED to use a range of input
parameters for the evaluated AQM (this may only be required if several
AQMs are being compared)
NK : The Figure 1 in section 3 MAY be used. The dumbbell topology is
useful to evaluate the performance of congestion control (such as
mentioned in the common TCP evaluation suite of the IRTF).
NK: Such as for the different parameters (buffer sizing, bottleneck
capacity, etc.), we understand that having unspecified parameters may
be annoying, but I do not see how we can define such parameters for
satellite networks, data-centers, home gateways, core Internet
providers’ network, etc. As one example, the capacity is (not only)
related to the mechanisms from lower layers, such as link layer
reliability schemes, channel access mechanisms, etc. which highly
varies from one context to the other. Also, the buffer sizing depends
on the BDP, the amount of traffic, the presence of PEP (such as in
satellite gateways), thus I am afraid we can hardly define a buffer
sizing that suits every use case. The amount of input parameters for
the evaluated AQM that is to be looked at is the amount of parameters
that are exhibited by the AQM developers (i.e. to what extent the AQM
is a black box or not). I hardly see how we can have guidelines that
specify number and parameterization that work for any use case
scenario and deployment context.
1. (section 5) several experiments with different TCP congestion
control schemes (+UDP) which should be performed on *unspecified*
bottleneck capacity and *unspecified* RTT, which should or should not
be the same as the one with which the AQM is configured.
NK: see previous comment.
2. (section 6) RTT fairness, where two groups of *unspecified* number
of flows share a bottleneck of *unspecified* capacity, where the first
group sees RTT of 100ms and the second is in range [5ms;560ms]
NK: see previous comment.
3. (section 7) burst absorption, is a group of four scenarios, which
include web-traffic, bursty video frames, which should be generated in
an unspecified manner (*left to the tester to decide*), CBR traffic of
*unspecified* rate, and TCP flow, all of which is evaluated using a
set of metrics that MAY be generated. I might be missing something,
but I don't understand how any of these metrics can be used to
characterize burst absorption.
NK : see previous comment.
4. (section 8) stability, including (a) varying the number of flows on
a bottleneck of *unspecified* capacity with *unspecified* RTT,
according to the provided formulas, (b) varying network capacity
between 10Mbps an 100Mbps for one TCP flow (if I understand it correctly)
NK: see previous comment.
5. (section 9) which includes (a) traffic mix of a combination
applications, including TCP, web-traffic, bi-direction VoIP using
*unspecified congestion* control (I am not sure, but I think there are
options), CBR of *unspecified* rate, and adaptive video streaming also
with *unspecified* congestion control, with one combination required
and other left to the tester to decide; and (b) bi-directional
traffic, which should evaluate the effect on dropping DNS/TCP Syn
packets* using a specified number of bulk TCP flows using the the
throughput-delay tradeoff graph; (in both scenarios capacities and RTT
are *unspecified* as well)
* I assume that DNS/SYN packets are mentioned in this section by mistake
NK: see previous comment for the rates. The comment on DNS and TCP Syn
packets have been introduced after Jim’s review
[https://mailarchive.ietf.org/arch/msg/aqm/pS9WlUlK5gExF8Ohu_syUXrNiUk ].
Do you think that a person without a substantial background knowledge
on the evaluation of AQM schemes can perform this evaluation (resolve
all unspecified conditions) in a reasonable amount of time?
NK: The unspecified parameters do not require specific knowledge on
AQMs – the unspecified parameters are (1) the RTT or the capacities,
which are related to the network underneath; (2) and the
characteristics of the traffic generation, which is related to the
traffic that is run between clients and servers. Thus, I think that a
person without a substantial background knowledge on the evaluation of
AQM schemes can perform this evaluation in a reasonable amount of time.
In the abstract, the document says that it describes characterization
guidelines for an AQM proposal, to decide whether it should be adopted
by the AQM WG. The WG currently has two AQMs (dropping/marking policy)
in last call. Did someone evaluate these AQMs according to the
specified guidelines?
Moreover, it seems to me that the WG is about to conclude. What
exactly is the purpose of standardizing this document then ?
NK: I am not sure whether the WG is about to conclude and do not have
such information. The purpose in standardizing this document as
INFORMATIVE would be to provide guidelines for evaluating AQM for the
specific issue of the bufferbloat. You may believe that the WG is
about to conclude, this draft may be useful for other initiative in
marking/dropping in routers.
Kind regards,
Nicolas
Regards,
Polina
On 12/01/2015 08:19 PM, Wesley Eddy wrote:
Thanks for making this update.
I don't think Roland and Polina who had made last call comments
had yet had the time to check that the changes in the last
revision met what they were expecting, so I'd like to give them
(and anyone else who has comments) a couple of weeks to check this
revision out, and assuming there are no major issues or objections
by around 12/15, will plan to end the working group last call, and
send the document to the area director for publication.
On 11/27/2015 5:50 AM, Kuhn Nicolas wrote:
Dear all,
This updated version integrates:
- modification on the buffer sizes, following some discussion
points raised by Michael Scharf on the tcpm mailing list [1];
- some nits raised by Greg Skinner.
Kind regards,
Nicolas KUHN
[1] https://www.ietf.org/mail-archive/web/tcpm/current/msg09894.html
-----Message d'origine-----
De : aqm [mailto:aqm-boun...@ietf.org] De la part de
internet-dra...@ietf.org <mailto:internet-dra...@ietf.org>
Envoyé : vendredi 27 novembre 2015 11:44
À : i-d-annou...@ietf.org <mailto:i-d-annou...@ietf.org>
Cc : aqm@ietf.org <mailto:aqm@ietf.org>
Objet : [aqm] I-D Action: draft-ietf-aqm-eval-guidelines-09.txt
A New Internet-Draft is available from the on-line Internet-Drafts
directories.
This draft is a work item of the Active Queue Management and
Packet Scheduling Working Group of the IETF.
Title : AQM Characterization Guidelines
Authors : Nicolas Kuhn
Preethi Natarajan
Naeem Khademi
David Ros
Filename : draft-ietf-aqm-eval-guidelines-09.txt
Pages : 35
Date : 2015-11-27
Abstract:
Unmanaged large buffers in today's networks have given rise to
a slew
of performance issues. These performance issues can be
addressed by
some form of Active Queue Management (AQM) mechanism,
optionally in
combination with a packet scheduling scheme such as fair queuing.
The IETF Active Queue Management and Packet Scheduling working
group
was formed to standardize AQM schemes that are robust, easily
implementable, and successfully deployable in today's
networks. This
document describes various criteria for performing precautionary
characterizations of AQM proposals. This document also helps in
ascertaining whether any given AQM proposal should be taken up
for
standardization by the AQM WG.
The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-aqm-eval-guidelines/
There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-09
A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-aqm-eval-guidelines-09
Please note that it may take a couple of minutes from the time of
submission until the htmlized version and diff are available at
tools.ietf.org.
Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/
_______________________________________________
aqm mailing list
aqm@ietf.org <mailto:aqm@ietf.org>
https://www.ietf.org/mailman/listinfo/aqm
_______________________________________________
aqm mailing list
aqm@ietf.org <mailto:aqm@ietf.org>
https://www.ietf.org/mailman/listinfo/aqm
_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm
