Hi Peter, thank you for your kind consideration of my notes. I have added a few new notes below under the GIM2>> tag. Attached, is the diff highlighting the updates resulting from our discussion.
Regards, Greg On Fri, Oct 13, 2023 at 9:13 AM <[email protected]> wrote: > Greg, > > > > Thanks for considering my review comments. My responses to specific points > are found below. > > > > Kind regards, > > -Peter > > > > *From:* Greg Mirsky <[email protected]> > *Sent:* Wednesday, October 11, 2023 1:54 AM > *To:* Peter Yee <[email protected]> > *Cc:* gen-art <[email protected]>; [email protected]; IETF > IPPM WG <[email protected]>; Last Call <[email protected]> > *Subject:* Re: Genart last call review of draft-ietf-ippm-pam-06 > > > > Hi Peter, > > thank you for your thorough review and detailed comments. Please find my > notes below under the GIM>> tag. I've attached the diff highlighting all > the updates. > > > > Regards, > > Greg > > > > On Tue, Oct 10, 2023 at 5:42 AM Peter Yee via Datatracker < > [email protected]> wrote: > > Reviewer: Peter Yee > Review result: Ready with Issues > > I am the assigned Gen-ART reviewer for this draft. The General Area > Review Team (Gen-ART) reviews all IETF documents being processed > by the IESG for the IETF Chair. Please treat these comments just > like any other last call comments. > > For more information, please see the FAQ at > > <https://wiki.ietf.org/en/group/gen/GenArtFAQ>. > > Document: draft-ietf-ippm-pam-06 > Reviewer: Peter Yee > Review Date: 2023-10-09 > IETF LC End Date: 2023-10-20 > IESG Telechat date: Not scheduled for a telechat > > Summary: This specification offers a scheme for metrics that are used in > determining if what you’re getting on a network is what you contracted > for. I > have some issues with the draft and the usual nits. [Ready with issues] > > Major issues: None > > Minor issues: > > General: For something that's Standards Track, I find the heavy use of > "can", > "could", and “might” leaving me wondering if this specification should be > Informational instead. It all seems a bit loose. You could do this or > that, if > you wanted… I get that the specific metrics are defined here, but even so, > there's not a lot of meat on the bone. > > GIM>> As the result of the discussion with our AD Martin Duke, the -07 > version of the document was set onto the Informational track > <https://mailarchive.ietf.org/arch/msg/ippm/E5FEuv5gQ3_rwotJCXpwascEsuQ/>. > In your opinion, the wording you've pointed out could be appropriate for an > informational document? > > > > PEY> The wording used and the lack of specificity made me think the > document was useful conceptually, giving guidance to the general idea of > PAMs but not something from which I would directly drive an implementation. > > > > > Page 4, 1st full paragraph, 2nd sentence: I've not followed this work, so > pardon me if I'm repeating a previous argument. However, I would find terms > like "compliance", "compliant", and "non-compliant" (maybe "incompliant") > preferable over "availability", "available", and "unavailable". Sure, you > get > PCM for an initialism instead of a nice acronym like PAM, but "available" > also > has a common meaning that makes it less desirable, in my opinion. > Compliance is > already used in the previous paragraph and seems like it could be > substituted > without a lot of drama. Of course, this will also affect section 3.3, > should > you choose the change the terms. > > GIM>> We, the authors, over the course of developing the document, > discussed the terminology among ourselves,e.g., the use of "conformance" > and "compliance", and with the WG. Current terminology, as I understand it, > is consistent and reflects the consensus of the WG. > > > > PEY> I’m completely fine with that and realize that I lack the knowledge > of he discussions that went on before. > > > > > Page 8, 2nd paragraph, 3rd sentence: I don’t find availability based on > successive intervals all that good a way of making that determination. A > service with an unavailability threshold set would be deemed "available" > if it > had a pattern of "unavailability threshold" - 1 interval violations > followed by > one VFI, rinse and repeat. Might you not want something like a percentage > of > VIs over a larger window of time instead? I realize this a couched in > permissive language, but then the following text runs with the idea that > successive intervals is the way to go. This also goes back to my general > issue > above about the looseness of this specification. > > GIM>> Thank you for the question. Violated Interval Ratio is one of the > PAM metrics that could be defined as noted in the last paragraph of Section > 3.3. That and other metrics can be used to improve the understanding of the > state of a service regarding the defined SLO thresholds. > > > > PEY> Thanks for that answer. I think the language in that paragraph also > reinforces my feeling that this draft is better as an Informational > specification. > > > > > Nits/editorial comments: > > Page 2, center header: I'm not sure how “PAM for Multi-SLO” applies as a > compression of the title. “Multi-SLO” never appears in any other context > than > the header of the document. > > GIM>> The intention of the shortened title was to highlight that PAM can > be particularly useful for services governed by more than one SLO, multiple > SLOs, as in the full title of the document - "Precision Availability > Metrics for Services Governed by Service Level Objectives (SLOs)". If you > find the short title not reflecting the intent of the document, could you > kindly suggest an alternative? > > > > PEY> Touché. I’m not entirely sure what I would use. It seemed to me that > the document wasn’t really emphasizing the multiplicity of SLOs, although > it certainly does use the plural for several terms such as parameter or > SLOs. On the other hand, it also works were there a single SLO being > measured. When reading the document, I wasn’t struck by a need for there to > be multiple SLOs or that there was any difference in whether one or > multiple SLOs were being measured. The SLOs seem to be independent > variables (at least at the high level of this document), so I was surprised > at the emphasis on the multiple part in the short title. Perhaps, > “Guidelines for PAMs”? > GIM2>> Thank you for the suggestion. It helped me a lot. What if the short name states "PAM Framework"? > > > Page 4, 1st full paragraph, 3rd sentence: delete “, and which” along with > “therefore” and if that parses better. Otherwise, the sentence feels like a > phrase. > > GIM>> That text has been edited in the current version -07: > > OLD TEXT in -06: > > "Precision" refers to the fact that services whose end-to-end service > > levels are governed by SLOs, and which must therefore be precisely > > delivered according to the associated quality and performance > > requirements. > > NEW TEXT in -07: > > "Precision" refers to services whose end-to-end service levels are > > governed by SLOs and must be delivered precisely according to the > > associated quality and performance requirements. > > Would you find the current version clearer and acceptable? > > > > PEY> Yes, that helps. > > > > Page 4, 2nd full paragraph, 1st sentence: change “is” to “are”. > > GIM>> Agreed. Also s/a separate topic/separate topics/ Right? > > > > PEY> Indeed! > > > > Page 5, section 3.1, 1st paragraph, 3rd sentence: “second” is given of an > example of a different granularity, but that’s the granularity given in the > second sentence, so it doesn’t make a good example of a different > granularity. > Decamillisecond is fine. > > GIM>> Removed "second or". Is that acceptable? > > > > PEY> Yes. > > > > Page 6, definitions for “VPC” and “SVPC”: How is a severely VPC different > from > a VPC? > > GIM>> Although, the applicability of these two metrics is the same, they > reflect aspects of different severity. > > > > PEY> Okay. > > > > It’s also not clear to me from the text that the performance parameters > for VI/SVI apply to individual packets, so how is this count generated? > Only > from parameters that can be applied on a per-packet basis (so not things > like > jitter)? > > GIM>> Indeed, some listed metrics in this list provided as examples of > what could be used. Our intention is to define new metrics in the future, > in a separate standard document. Do you see that as a reasonable approach > for the informational document or would suggest another path forward? > > > > PEY> That approach is fine to me. > > > > Page 7, 6th and 7th bullet items: I’d change “measurement interval” to > “measurement session” (session was used previously) or “period”. > Otherwise, the > term “interval” becomes unnecessarily overloaded. > > GIM>> I think that using "interval" in these sentences is intentional to > reference VI, SVI, and VFI. Could you kindly suggest how we can make it > clearer? > > > > PEY> My argument would be for “measurement session”, as used in the first > paragraph of 3.1. Interval is used here as the period for a VI, SVI, or > VFI. I see the session as the period covering all of the intervals. > GIM2>> Thank you for the clarification. I agree. Updated text as you've suggested. > > > Id-nits says: > > == There is 1 instance of lines with non-ascii characters in the > document. > > > I’m not sure where that character is and I admit to not searching for it. > > GIM>> It appears that the warning is triggered by 'ø' in the following in > Acknowledgment: > > The authors greatly appreciate review and comments by Bjørn Ivar > > I hope that this is tolerable. > > > > PEY> Completely. I just couldn’t easily find (on the particular OS I was > using) what was causing idnits to complain. >
<<< text/html; charset="US-ASCII"; name="draft-ietf-ippm-pam-08.diff.html": Unrecognized >>>
_______________________________________________ Gen-art mailing list [email protected] https://www.ietf.org/mailman/listinfo/gen-art
