Re: [alto] Artart last call review of draft-ietf-alto-performance-metrics-17
Hello Richard,
On Mon, Oct 18, 2021 at 03:43:50PM -0400, Y. Richard Yang wrote:
> Good comment. The document gives the high-level grammar (1 line) at the
> beginning of Sec. 2.2.
> It looks that your suggestion is the write out the complete grammar upfront:
I was not so much looking for the details of the terms "where
MUST be one of the following" was fine) but for
a) naming the formal language this is in. Reading, I've guessed that the
square brackets mean "optional" because I've seen that in DOS-time
documentation, and it looks formal, but does not declare a language.
A formal language that could be used here and is common in RFCs would
be ABNF.
b) the tying in of the output -- the the term "metric-identifier" is not
used anywhere else, and RFC7285 uses a CostMetric type in the
cost-metric field; I'd appreciate if these identifiers were somehow
linked.
This would be trivial had ALTO used CDDL to describe its JSON, as
then there could be a line like `cost-metric //= metric-identifier`;
maybe the `object { ... }` notation has something similar?
RFC7285 generally does without constructing string identifier, so
there is no ABNF in there.
If you go with the suggested colon structuring and have text for that,
possibly there the optionality and string concatenation be described
in ABNF (or just in words with an example), and the need for a formal
language would go away here.
> "... Note that some systems use quantile, which is in the range [0, 1].
> This document uses percentile to make the identifier easier to read. When
> there is a more common form for a given percentile, it is RECOMMENDED that
> the common form being used; that is, instead of p0, use min; instead of
> p50, use median; instead of p100, use max".
Fine with me.
> > * Allowing decimals into the cost metric identifier introduces a dot which
> > is reserved as per RFC7285 Section 10.6.
>
> Yes. Correct. From the grammar above, we made sure that we would not
> introduce a dot. We can add a sentence to point out that when choosing the
> base, we should follow this. Should we do this?
I think it'd be easiest (with the grammar or without) to to introduce
the number as a nonnegative integer. (That'd allow dropping mentions of
the minus and exp components).
> > A way out could be to formalize this structure and register the
> > metric-base-identifiers for use with and without a stat parameter
> > following the colon (instead of the dash).
>
> So the suggestion is that ":" as the consistent internal structure
> separator. This is quite reasonable.
Just beware that this is formally an update to the registry; not sure
whether that incurs an "updates" to 7285.
> > A few words on which statistic can be used with which metric could
> > also help with bw-maxres. (What does bw-maxres-p50 mean, is it
> > meaningful at all?)
>
> Mathematically, any percentile of a set of a single value is the value
> itself. But it is indeed a good idea to clarify it. We will add a sentence
> at the end of 2.2
>
> => Note that although one can use generic statistics (i.e., any percentile
> in [0, 100]) and multiple specifications may give the same value, it helps
> to choose the more intuitive and robust definition. For example, when the
> set is expected to be a single value. The max operator is more robust and
> hence recommended.
That sounds more confusing to me (the max operator ... so I should use
bw-maxres-max?); maybe (if correct) something like
| Note that unlike the other metrics, maxres is a single value and not
| sampled over time. Thus, it MUST NOT be specialized with a stat
| indicator, but is to be used in the base form.
> > * "Content-Length: TBA": Does this add value to the examples?
>
> We will add the final exact value when publishing, as it is part of HTTP
> header.
Sure it is part of the HTTP header, but so are many other headers
(Accept-Content-Encoding, User-Agent, what so not). Accept and
Content-Type add to the understanding, Content-Length will IMO be
ignored by all readers anyway.
> It is an example to illustrate both ipv4 and ipv6. If we do ipv4->ip4 and
> ipv6-ipv6, we will need two examples, and it takes more space.
Does it need examples of all of them? These are not unit tests, they are
illustrative.
> > * "the - component": "any 'stat' component"?
>
> Not sure what this is? Any more specific locator?
3.1.3
> Very good comment! We have made the changes. The newer version fixed this
> issue.
> Please see -18 which is just loaded today.
Didn't see that at first ... something was changed in the uplaod process
and now there's no HTML version any more?
BR
c
--
To use raw power is to make yourself infinitely vulnerable to greater powers.
-- Bene Gesserit axiom
signature.asc
Description: PGP signature
___
alto mailing list
alto@ietf.org
https://www.ietf.org/mailman/listinfo/alto
Re: [alto] [art] Artart last call review of draft-ietf-alto-performance-metrics-17
Hello, On Mon, Oct 18, 2021 at 07:02:08AM -0700, Christian Amsüss via Datatracker wrote: > ## Summary for the IoT Directorate I performed the review with the wrong hat on, sorry for the mixup. Refocusing on the ART review criteria brings up nothing new -- but the point about the formal language used with metric-identifier deserves more emphasis. Likewise, the point about registration of the stat-typed metric identifiers (and the possible structuring of the registry into registered prefixes and per-prefix semantics for what is behind a colon) now falls more directly into the scope of the review. Otherwise, what was said with focus on the IoT applies likewise to the use of HTTP and other general ART topics: > As for conventions around the Internet of Things, this makes no choices -- if > follows the path set out by ALTO, and adds terms and considerations for > metrics > established outside of ALTO. Best regards Christian -- To use raw power is to make yourself infinitely vulnerable to greater powers. -- Bene Gesserit axiom signature.asc Description: PGP signature ___ alto mailing list alto@ietf.org https://www.ietf.org/mailman/listinfo/alto
[alto] Artart last call review of draft-ietf-alto-performance-metrics-17
Reviewer: Christian Amsüss Review result: Ready with Issues ## Summary for the IoT Directorate This document extends the Application-Layer Traffic Optimization (ALTO) protocol, by which applications that need to a peer from a selection can ask their uplink for preferences. It adds more fine-grained vocabulary to pick the peer with best expected bandwidth, or best latency. As for conventions around the Internet of Things, this makes no choices -- if follows the path set out by ALTO, and adds terms and considerations for metrics established outside of ALTO. The mechanisms (more of ALTO than this particular document) can be applicable to IoT applications when unconstrained devices relay data into cloud systems -- then they might make better choices in presence of decentralized backends. If this document gains traction in deployments, these systems will need deeper application knowledge pertaining to the application's requirements (selecting the low-latency vs. the high-bandwidth option). ## High-level The document can be followed well after brief familiarization with ALTO; it is in a good over-all shape. 2.2 Performance Metric Statistics: * The metric-identifier definition seems disconnected with the rest of the terminology. I assume from context that this is a way for building a CostMetric value. If this assumption is wrong, some of the later points are moot. (By the way, what language is the metric-identifier definition? It's not ABNF, and I don't find any other formal language in thenormative references.) * p0 and p100 are aliases to min and max IIUC. Is there a preferred form, and why is the other form allowed? (Same for p50 and median.) * Allowing decimals into the cost metric identifier introduces a dot which is reserved as per RFC7285 Section 10.6. * comparing to 7 IANA Considerations: The registered identifeirs are only the metric-base-identifiers. Registering all combined values from metric-base-identifier x stat is not practical (especially with the numeric percentiles); however, the 'priv:' prefix indicates that some structure was intended in RFC7285. A way out could be to formalize this structure and register the metric-base-identifiers for use with and without a stat parameter following the colon (instead of the dash). * Section 2.2 (Performance Metric Statistics) allows adding -stdvar to metric-base-identifiers; delay-variation-mean is probably similar to delay-ow-stdvar. It's only similar (and not identical) due to the offset from minimum detailed in 3.3.3; anyhow, pointing out that out in the "Note that in statistics" paragraph, e.g. as in "... networking convention. Due to this, it is expressed as a dedicated metric and not just as delay-ow-stddev". A few words on which statistic can be used with which metric could also help with bw-maxres. (What does bw-maxres-p50 mean, is it meaningful at all?) 3 Packet Performance Metrics: * Cost-Context Specification Considerations: There is a lot of duplication here, especially around SLA and the topic of "link" descriptions. (For the technically interesting parts, the later sections already refer back to 3.1.4). 6 Security Considerations: * On the topic of confidentiality, can these metrics be used with the "ordinal" cost mode? A client might just want to pick the peer that has the "best" round-trip time, and could thus avoid the need to obtain the confidential metrics. ## Nits * Table 1: The current XML RFC format can do tables that render to HTML, this would also make it easier to follow the links. For tables and figures, also more accessible labeling is available there. * Section 3 Packet Performance Metrics: The pkt.* notation appears very ad-hoc, and is not used anywhere outside the paragraph. If these are referring to external definitions, please consider referencing that definition; otherwise: do these labels add value? * "smaller than milliseconds": smaller than one millisecond? * "Content-Length: TBA": Does this add value to the examples? * Example in 3.1.3: Does the example make sense in terms of addresses? Querying metrics between an IPv4 and an IPv6 address seems strange. (And is this primarily used in a field where IPv4 is still dominant?) In particular, the hop count example has an unreasonably high number of hops between 192.0.2.2 and 192.168.2.89... * "the - component": "any 'stat' component"? ## Slightly off topic (ie. this is out my expertise and would be checked later by someone else, but it may be helpful to revisit it right away) * IANA considerations: Creating a new regisry usually comes with ore than "reqeusts the creation of" and some values; https://datatracker.ietf.org/doc/html/rfc8126#section-2.2 in particular asks for a registration policy. (Given the document says "
