1. One problem here is that 3649 and 8312 aren't in the references at all, so that should be fixed if we retain the text
2. IMO collection of bandwidth information is out of scope of this document; in general collection of network information is out of scope for ALTO. I viewed the references as informative examples rather than a prescription of how it's collected. 3. I agree, in retrospect, that 3649 and 8312 are bad suggestions, as their result is very time-dependent and noisy. From the rest of section 4, this metric is clearly in the context of bandwidth reservations and min(raw link capacity) is probably a better indicator. However, the rest of these sections don't really talk about metric definitions, so it might be best simply to delete the parenthetical. So my suggestion would be to take Lars's request to define what the authors mean a little precisely (raw capacity?), but not to the granularity that IPPM defines metrics, and get rid of the references entirely. Is that satisfactory to you, Lars? Martin On Mon, Nov 29, 2021 at 5:10 AM Lars Eggert via Datatracker < [email protected]> wrote: > Lars Eggert has entered the following ballot position for > draft-ietf-alto-performance-metrics-19: Discuss > > When responding, please keep the subject line intact and reply to all > email addresses included in the To and CC lines. (Feel free to cut this > introductory paragraph, however.) > > > Please refer to https://www.ietf.org/blog/handling-iesg-ballot-positions/ > for more information about how to handle DISCUSS and COMMENT positions. > > > The document, along with other ballot positions, can be found here: > https://datatracker.ietf.org/doc/draft-ietf-alto-performance-metrics/ > > > > ---------------------------------------------------------------------- > DISCUSS: > ---------------------------------------------------------------------- > > This document needs to become much more formal about how it defines the > metrics it wishes to use with ALTO. This could either be done either by > identifying and normatively referencing existing metrics the IETF has > defined, > or by defining them here. When normatively referencing existing IETF > metrics, it > would need to explain why their use with ALTO makes sense. > > At the moment, the document informatively points to a somewhat arbitrary > collection of prior IETF metrics (most of which are from IPPM, residual > bandwidth from IS-IS TE, but then reservable bandwidth from OSPF TE?). But > it > only refers to them as "examples", without actually defining how exactly > they > are to be used with ALTO, or - if not those - which actual metrics are > supposed > to be used. > > Defining a mechanism for exposing metric information to clients isn't > really > useful unless the content of that information is much more clearly > specified. > > Section 4.1.3. , paragraph 2, discuss: > > Intended Semantics: To give the throughput of a TCP congestion- > > control conforming flow from the specified source to the specified > > destination; see [RFC3649, Section 5.1 of RFC8312] on how TCP > > throughput is estimated. The spatial aggregation level is specified > > in the query context (e.g., PID to PID, or endpoint to endpoint). > > A TCP bandwidth estimate can only be meaningfully be derived for bulk TCP > transfers under a set of pretty strict and simplistic assumptions, making > this > metric a meaningless at best and misleading at worst, given that the > source of > this information doesn't know what workload, congestion controller and > network > conditions the user of this information will use or see. > > Also, RFC3649 is an Experimental RFC (from 2003!) and RFC8312 is an > Informational RFC. Since this document normatively refers to them, it > needs to > cite them, and this will cause DOWNREFs for PS document. I would argue that > at least RFC3649 is certainly not an appropriate DOWNREF. > > Why define this metric at all? The material you point to is the usual > model-based throughput calculation based on RTT and loss rates; a client > that > intended to predict TCP performance could simply query ALTO for this and > perform > their own computation, which will likely be more accurate, since the > client will > hopefully know which congestion controller they will use for the given > workload, > and what the characteristics of that workload are. > > > ---------------------------------------------------------------------- > COMMENT: > ---------------------------------------------------------------------- > > Section 1. , paragraph 6, comment: > > The purpose of this document is to ensure proper usage of the > > performance metrics defined in Table 1; it does not claim novelty of > > the metrics. The "Origin Example" column of Table 1 gives an example > > RFC that has defined each metric. > > I don't understand what the purpose of the "origin example" column is. > Most of > these point to IPPM metrics, which have a pretty clear and > narrowly-defined area > of applicability. Since ALTO isn't performing IPPM-style network testing, > it's > not clear why IPPM metrics are referenced here? > > Section 2.2. , paragraph 23, comment: > > If a cost metric string does not have the optional statical operator > > string, the statistical operator SHOULD be interpreted as the default > > statical operator in the definition of the base metric. If the > > What is a "statical" operator; I am not familiar with the term and it > doesn't > seem to appear in other RFCs? (Also occurs elsewhere in this document.) > > Section 3.1.4. , paragraph 4, comment: > > link statistics. Another example of a source to estimate the delay > > is the IPPM framework [RFC2330]. It is RECOMMENDED that the > > IPPM defines measurement metrics. How would they be a source for estimates? > > Section 3.3. , paragraph 1, comment: > > 3.3. Cost Metric: Delay Variation (delay-variation) > > Is this supposed to apply to the one-way or bidirectional delay? Also, > delay > variation is not independent from path utilization (c.f. bufferbloat), so > why is > it being reported independently? > > Section 3.5. , paragraph 1, comment: > > 3.5. Cost Metric: Loss Rate (lossrate) > > What is this metric supposed to capture? Loss is generally not independent > from > network utilization (apart from random corruption loss). So it should be > zero > for unloaded networks, and depends on utilization otherwise. Also, is this > unidirectional or bidirectional loss (wording below is unclear)? > > Using lowercase "not" together with an uppercase RFC2119 keyword is not > acceptable usage. Found: "MUST not" > > The document has 6 authors, which exceeds the recommended author limit. I > assume the sponsoring AD has agreed that this is appropriate? > > No reference entries found for: [RFC3649] and [RFC8312]. > > Found terminology that should be reviewed for inclusivity; see > https://www.rfc-editor.org/part2/#inclusive_language for background and > more > guidance: > > * Term "man"; alternatives might be "individual", "people", "person". > > > ------------------------------------------------------------------------------- > All comments below are about very minor potential issues that you may > choose to > address in some way - or ignore - as you see fit. Some were flagged by > automated tools (via https://github.com/larseggert/ietf-reviewtool), so > there > will likely be some false positives. There is no need to let me know what > you > did with these suggestions. > > "Abstract", paragraph 2, nit: > - types of cost metric. Since the ALTO base protocol (RFC 7285) > + types of cost metrics. Since the ALTO base protocol (RFC 7285) > + + > > Section 1. , paragraph 2, nit: > > ] on registering ALTO cost metrics. Hence it specifies the identifier, > the in > > ^^^^^ > A comma may be missing after the conjunctive/linking adverb "Hence". > > Section 2.2. , paragraph 2, nit: > > of the observations. median: the mid point (i.e., p50) of the > observations. > > ^^^^^^^^^ > This word is normally spelled with a hyphen. > > "IPPM ", paragraph 2, nit: > > Also, delay variation is not independent from path utilization (c.f. > buffer > > ^^^^^^^^^^^^^^^^ > The usual collocation for "independent" is "of", not "from". Did you mean > "independent of"? > > Section 3.3.3. , paragraph 7, nit: > > apture? Loss is generally not independent from network utilization > (apart fr > > ^^^^^^^^^^^^^^^^ > The usual collocation for "independent" is "of", not "from". Did you mean > "independent of"? > > Section 3.4.3. , paragraph 6, nit: > > imation" method. See Section 3.1.4 on on related discussions such as > summing > > ^^^^^ > Possible typo: you repeated a word. > > Section 3.5.4. , paragraph 3, nit: > > [RFC8312]), it helps to specify as much details as possible on the the > cong > > ^^^^ > Use "many" with countable plural nouns like "details". > > Section 3.5.4. , paragraph 3, nit: > > ify as much details as possible on the the congestion control algorithm > used > > ^^^^^^^ > Two determiners in a row. Choose either "the" or "the". > > These URLs in the document can probably be converted to HTTPS: > * > http://www.iana.org/assignments/alto-protocol/alto-protocol.xhtml#cost-metrics > > > >
_______________________________________________ alto mailing list [email protected] https://www.ietf.org/mailman/listinfo/alto
