Hi Paul, Thank you very much for taking the time to review the draft!
You are right in that the measurements / reported percentiles of different network segments need to be the same to enable composability. Ensuring this alignment across different networks / operators might be challenging, especially considering that the draft does not impose standardized percentiles for the network performance requirements of applications. The draft does follow BITAG recommendations regarding measurement percentiles, though, and I think that once we have a better understanding which measurement percentiles have the most meaning for applications (with application “profiles" then also using those percentiles), alignment across operators could very well just follow naturally. Even if it does not follow, there is still a lot of value to be gained by just using the methodology on different network segments within the same network. I believe we already have some limited (implicit) text along this line in the document. If you see value in clarifying possible deployment / adoption challenges beyond what we currently have, we could add some additional, explicit discussion focusing on this aspect in the operational consideration section. What do you think? Regarding the presentation of the formulas, you have probably also seen in the thread with Martin that we plan to (at the very least) implement his suggestion that you also refer to in your comment. Additionally, Martin has kindly provided me additional pointers for alternative forms of formula rendering and we plan to also explore these options to come up with a version that is as clear and easy to understand as possible. I would very much appreciate your feedback on the final version once we have it. Thank you again! Cheers, Ike Friendly Note: I send emails at times that work best for me. Please don’t feel any pressure to reply outside your usual working hours. [Background pattern Description automatically generated] Ike Kunze Senior Researcher Phone +47 408 81 254 CUJO AI<https://cujo.com/> | News <https://cujo.com/blog/> | LinkedIn<https://www.linkedin.com/company/cujoai> From: Paul Kyzivat <[email protected]> Date: Monday, 9 February 2026 at 16:38 To: [email protected] <[email protected]> Cc: General Area Review Team <[email protected]>, [email protected] <[email protected]>, [email protected] <[email protected]> Subject: Gen-ART Last Call review of draft-ietf-ippm-qoo-06 CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. I am the assigned Gen-ART reviewer for this draft. The General Area Review Team (Gen-ART) reviews all IETF documents being processed by the IESG for the IETF Chair. Please treat these comments just like any other last call comments. For more information, please see the FAQ at <https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.ietf.org%2Fen%2Fgroup%2Fgen%2FGenArtFAQ&data=05%7C02%7Cike.kunze%40cujo.com%7Cc05c24a2dabe489aaf4f08de67f1347e%7Ca5d2f1ce8ca649ce893ab4f922d7fc6b%7C0%7C0%7C639062482918728250%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=UPw6Ih9qEWczjMlQslXsnJu%2BtkPYy77aFYjavK%2FfSrs%3D&reserved=0<https://wiki.ietf.org/en/group/gen/GenArtFAQ>>. Document: draft-ietf-ippm-qoo Title: Quality of Outcome (QoO) Reviewer: Paul Kyzivat Review Date: 2026-02-09 IETF LC End Date: 2026-02-13 IESG Telechat date: ? Summary: This draft is on the right track but has open issues, described in the review. Note: This reviewer has no experience in, or knowledge of, network performance measurement. Hence I'm unqualified to comment on the semantics of this draft. In such cases I usually fall back to reviewing the form and syntax. Even that is hard here, since there is little in the way of formal specification. Issue: My impression is that for QoO values to be composable, they must be based on the same measurement percentiles. Achieving that across the managers of connected networks seems difficult. Is it realistic? Issue: I had difficulty understanding the notation used in the formula in section 7: QoO_latency = min_{i}(min( max((1 - ((ML_i - ROP_i) / (CPUP_i - ROP_i))) * 100, 0), 100)) Fortunately, Martin Thompson has posted an alternative rendition for this formula, that I find much more understandable: for i in ROP: m = (ML[i] - ROP[i]) / (CPUP[i] - ROP[i]) metrics[i] = clamp(0, m, 1) QoO_latency = find_min(metrics) * 100 (It is in his followup message to his ArtArt review of this document. <https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailarchive.ietf.org%2Farch%2Fmsg%2Fart%2FBC-qxC6pD_63lfHfLd1R4n_KA8c%2F&data=05%7C02%7Cike.kunze%40cujo.com%7Cc05c24a2dabe489aaf4f08de67f1347e%7Ca5d2f1ce8ca649ce893ab4f922d7fc6b%7C0%7C0%7C639062482918760940%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=RWF9tZEOYG5pbms6GqROHFbOI%2BO7L9dQlRIvxCltmrY%3D&reserved=0<https://mailarchive.ietf.org/arch/msg/art/BC-qxC6pD_63lfHfLd1R4n_KA8c/>>) Martin considers this a nit, but I consider it to be a significant issue. It is the closest thing this document has to a normative requirement, so it is essential that all readers understand it consistently. I recommend reworking the rendition of this formula into a form that will be clear to all who need to understand it. I found Martin's full ArtArt review insightful. While it goes beyond my understanding of the subject material, I agree with it to the extent of my understanding. This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
img-2458bd2e-7d8c-4793-944f-7d2451498a8f
Description: img-2458bd2e-7d8c-4793-944f-7d2451498a8f
_______________________________________________ Gen-art mailing list -- [email protected] To unsubscribe send an email to [email protected]
