Hi Curtis, Thank you for the detailed response. I have included specific responses in-line below. Before we go there, I'd like to jump ahead to one point that is (indirectly) pointed to in a number of spots.
I asked: >> So where does one go to get an intro/definition of the IETF work on >> composite links? Is it G.800, the framework draft, the use cases >> draft, this draft? ... Your response is: > I'd say "Use Cases". I'm in favor of de-emphasizing G.800. Okay, then I think a bunch of detailed comments can be replaced by using references. More on this below. The issue with the use case document being the authoritative definition for CL (and presumably NPO?) is that I think you currently have circular references. draft-ietf-rtgwg-cl-use-cases starts out: Composite link requirements are specified in [I-D.ietf-rtgwg-cl-requirement]. A composite link framework is defined in [I-D.ietf-rtgwg-cl-framework]. BTW ietf-rtgwg-cl-framework adds Classic multipath, including Ethernet Link Aggregation has been widely used in today's MPLS networks [RFC4385][RFC4928]. Classic multipath using non-Ethernet links are often advertised using MPLS Link bundling. A link bundle [RFC4201] bundles a group of homogeneous links as a TE link to make IGP-TE information exchange and RSVP-TE signaling more scalable. A composite link allows bundling non-homogenous links together as a single logical link. The motivations for using a composite link are descried in [I-D.ietf-rtgwg-cl-requirement] and [I-D.ietf-rtgwg-cl-use-cases]. A composite link is a single logical link in MPLS network that contains multiple parallel component links between two MPLS LSR. Unlike a link bundle [RFC4201], the component links in a composite link can have different properties such as cost, capacity, delay, or jitter. I think answering this question (of which document has the CL definition) and ensuring the documents line up accordingly would be really helpful. -- I'll assume it's the use case draft for my comments below. At the nit level, the defining document should be listed as a Normative Reference by this (and other CL) documents. On 6/20/2013 3:51 PM, Curtis Villamizar wrote: > In message <[email protected]> > Lou Berger writes: > >> Hello, >> >> I have been selected as the Routing Directorate reviewer for this draft. >> The Routing Directorate seeks to review all routing or routing-related >> drafts as they pass through IETF last call and IESG review, and >> sometimes on special request. The purpose of the review is to provide >> assistance to the Routing ADs. For more information about the Routing >> Directorate, please see http://www.ietf.org/iesg/directorate/routing.html >> >> Although these comments are primarily for the use of the Routing ADs, it >> would be helpful if you could consider them along with any other IETF >> Last Call comments that you receive, and strive to resolve them through >> discussion or by updating the draft. >> >> Document: draft-ietf-rtgwg-cl-requirement-10 >> Reviewer: Lou Berger >> Review Date: 6/19/2013 >> IETF LC End Date: 2013-06-19 >> Intended Status: Informational > > Hi Lou, > >> Summary: >> I have significant concerns about this document and recommend >> that the Routing ADs discuss these issues further with the authors. > > OK. Lets go through them. > >> Comments: >> >> My comments really fall into three categories: (a) understanding >> document objectives, (b) comments/questions on technical details, and >> (c) readability/editorial. As this document is informational and does >> not represent IETF consensus, the AD together with the document Shepherd >> and WG may reasonably choose to publish the document as is. > > Does not reflect IETF consensus? > Sure, see http://tools.ietf.org/html/rfc5741, informational documents can represent IETF consensus, but don't need to; it depends on LC and is represented in the boilerplate. The datatracker represents this and says "Consensus: says unknown". I translated this into no request for IETF consensus was to be made -- I accept that this may be a mistake on may part. (Perhaps it was wishful thinking ;-) The document Shepherd should be able to answer this. Alia? >> Major Issues: >> >> - At the highest level the objective/purpose of the document is unclear >> to me. It states: >> >> The purpose of this document is to describe why network operators >> require certain functions in order to solve certain business problems >> (Section 2). > > Forward reference for brevity in "1. Introduction". Is that bad? Brevity is great, but what's that quote "As brief as possible (but no briefer)"... I genuinely am confused as to the purpose of the document so it's my conclusion that the Intro is too brief. Other conclusions are of course possible. > > Section 1 is brief and forward references sections 2, 4, 5, and > appendix A. > >> Thankfully (for hopefully obvious reasons), section 2 covers >> "Assumptions" and not "business problems". Section 4 does provide what >> is termed "Network Operator Functional Requirements", which relies on >> Y.1541 defined Network Performance Objectives. While this section title >> appears to be defining provider requirements, the document also appears >> to be placing requirements on CL solutions. To add to the confusion, it >> sometimes also seems to also be placing requirements on the "network >> layer" that supplies component links for the CL, and even on the >> performance of a vendor implementation(DR#7)! > > It seems the objection is to the phrase "business problems". If we > change the first sentence in section 1, would that solve it? > > OLD > > The purpose of this document is to describe why network operators > require certain functions in order to solve certain business > problems (Section 2). > > NEW > > The purpose of this document is to describe why network operators > require certain functions and to clearly enumerate a set of > requirements related to the use of MPLS based Composite Links > (see Section 2). Well, I see a couple of phrases that confuse me in the context of the document. Certainly the phrase "... to solve certain business problems (Section 2)" caused me to be a bit concerned that an IETF document was going to discuss business issues; it also led me to believe that section 2 was going to discuss these problems. As I said, there are no "business problems" covered in section 2, so aligning the Intro with the section contents is a fine improvement. The other phrase that confuses me is "... describe why network operators require certain functions ... (See Section 2)". Section 2 is titled Assumptions and only references network operators to say that "...in general network operators do not rely on the DSCP...". So I think your revised text isn't really aligned with the section. Finally, you now say the draft documents requirements on the "use of MPLS based Composite Links". So this reads like the document places requirements on the user, i.e., the operator. Do you perhaps mean "on the protocols and mechanisms used to provide MPLS based Composite Links"? Taken all together, perhaps the following better matches the/your intent. The purpose of this document is to clearly enumerate a set of requirements related to the protocols and mechanisms that provide MPLS based Composite Links. MPLS based Composite Links are defined in [I-D.ietf-rtgwg-cl-use-cases]. Section 2 describes MPLS based Composite Links assumptions. > > Then in section 2, change "The services supported include ..." to > "Services which may be supported over MPLS based Composite Links > include ...". That is helpful. Do you perhaps mean: "Any MPLS-based service may be supported over MPLS based Composite Links including, for example, ..." > >> I think clearly stating the document's objective/purpose and where the >> stated requirements are to be applied/satisfied, is one of major issues >> that should be corrected. >> >> Once this is done, it may turn out that there are derivative >> comments/changes that are needed in the body of the draft. For example >> clarifying what it means to "meet the NPO" and who needs to "meet" it. > > The reason the term NPO is used it because of objections to using the > term SLA. A Service Level Agreement (SLA) is interpreted as > contractual. A Network Performance Objection (NPO) may simply be an > interal goal that an organiztion would like to meet. The NPO may or > may not be based on Y.1540 and Y.1541. The term NPO is defined in > Y.1541 and the use of the term in this docuement is consistent with > that definition. > > The use cases document has a long paragraph defining SLA, SLS, and > NPO. Those definitions are in an appendix intended to be illustrative > and entitled "More Details on Existing Network Operator Practices and > Protocol Usage". The use case document also says: In this document we use the term Network Performance Objective (NPO) as defined in section 5 of [ITU-T.Y.1541] since the SLA and SLS measures have network operator and service specific implications. > > The definition here is simply: > > Network Performance Objective (NPO): Numerical values for > performance measures, principally availability, latency, and > delay variation. See [I-D.ietf-rtgwg-cl-use-cases] for more > details. > > I can expand on this and say "NPO may or may not be based on ITU > specifications." I prefer not to get too wordy when all we are trying > to say "like SLA or SLS (in diffserv), but NPO is really a better > term". > Why not just say: Network Performance Objective (NPO): as defined by [I-D.ietf-rtgwg-cl-use-cases]. > As to what it means by "meet the NPO", it means "falls within the > numerical values for measures, principally availability, latency, and > delay variation". I'm not sure why that is not obvious. Try using this expansion in the identified requirements and see if you think it makes sense in all cases. Also, recall that Table 1 in section 5 of [ITU-T.Y.1541] provides absolute time values based on traffic "QoS class" and that a number of values are also specified as "U", which is stated as meaning U", performance with respect to that parameter may, at times, be arbitrarily poor. For example: FR#1 The solution SHALL provide a means to summarize some routing advertisements regarding the characteristics of a composite link such that the routing protocol converges within the timeframe needed to [meet the network performance objective] fall within the numerical values for measures, principally availability, latency, and delay variation FR#2 The solution SHALL ensure that all possible restoration operations happen within the timeframe needed to [meet the NPO] fall within the numerical values for measures, principally availability, latency, and delay variation I really think more is needed to make these requirements meaningful in the design of any solution. For example, I could see something along the lines of the following as being meaningful: (Note this text is for discussion not inclusion in the draft.) FR#2 The solution SHALL enable restoration operations to occur within the timeframe need to provide an Upper bound on the packet loss probability (NPO IPLR) of 1x10^-3 in a network with a diameter of XYZ. Do you think I'm wrong? On an aside, how do FR#1 and DR#6 really differ? DR#6 seems to be the better formulated to me. > > As to the "who", it is the provider but the protocols and > implementation have to provide the means for the provider to do so. I think this point is now understood, assuming there's no substantive disagreement on the revised intro. > As to the scope of the NPO: if the NPO is defined over a link, the > link needs to meet it; if it is defined as edge-to-edge within an > internal infrastructure, it must be met edge-to-edge; if it is defined > for a specific customer end-to-end, then it must be met end-to-end > (and may correlate strongly with a specific SLA). Humm, the mechanisms needed to satisfy timing requirements between adjacent nodes can be radically different (simpler) than between network paths. I'm not sure NPO-based requirements can be meaningful without some design targets/constraints. At least ITU-T.Y.1541 provides some absolute time values. > > At what point does clarifying what is meant by an NPO and what it > means to meet an NPO just overly wordy and stating the obvious? I think this is covered above. > >> - The scope of the composite links referenced in this document is not >> clear. The document seems to be using G.800 terminology and concepts, >> yet the G.800 related summary is relegated to an Appendix. If this CL >> is being driven/constrained by the G.800 CL, then this relationship >> should be made explicit and in the (early) body of the document, or by >> reference to another document that defines CL as such. (In other words >> Appendix A should early in the document, perhaps in section 1.) > > We give a definition for composite link and component link. ITU gives > a definition for the same two terms. We have give a summary of a few > ITU-T G.800 definitions in Appendix A. I would be perfectly happy > getting rid of Appendix A and simply being more clear in our > definitions section. > > OLD > > ITU-T G.800 Based Composite and Component Link Definitions: > Section 6.9.2 of ITU-T-G.800 [ITU-T.G.800] defines composite and > component links as summarized in Appendix A. The following > definitions for composite and component links are derived from > and intended to be consistent with the cited ITU-T G.800 > terminology. > > NEW > > ITU-T G.800 Based Composite and Component Link Definitions: Section > 6.9.2 of ITU-T-G.800 [ITU-T.G.800] defines composite and > component links. The following definitions for composite and > component links are derived from and intended to be consistent > with the cited ITU-T G.800 terminology, but differ slightly. > > We can then point out that we are specifically excluding inverse > multiplexing from our definition of Composite Link and that we are not > using the ITU definition. > > I would like to see us use the definition of multipath in > draft-ietf-mpls-multipath-use but this work started out using the term > composite link so we may be stuck with it. Some key definitions in > draft-ietf-mpls-multipath-use are: > > Multipath > The term multipath includes all techniques in which > > 1. Traffic can take more than one path from one node to a > destination. > > 2. Individual packets take one path only. Packets are not > subdivided and reassembled at the receiving end. > > 3. Packets are not resequenced at the receiving end. > > 4. The paths may be: > > a. parallel links between two nodes, or > > b. may be specific paths across a network to a destination > node, or > > c. may be links or paths to an intermediate node used to > reach a common destination. > > Composite Link > The term Composite Link had been a registered trademark of Avici > Systems, but was abandoned in 2007. The term composite link is > now defined by the ITU in [ITU-T.G.800]. The ITU definition > includes multipath as defined here, plus inverse multiplexing > which is explicitly excluded from the definition of multipath. > > Inverse Multiplexing > Inverse multiplexing either transmits whole packets and > resequences the packets at the receiving end or subdivides > packets and reassembles the packets at the receiving end. > Inverse multiplexing requires that all packets be handled by a > common egress packet processing element and is therefore not > useful for very high bandwidth applications. > > We can also add the following: > > Classic Multipath > Classic multipath refers to the most common current practice in > implementation and deployment of multipath where multipath > consist only of homogeneous links and in cases such as Ethernet > Link Aggregation are entirely transparent to upper layer > control planes. > > These IMHO are more clear definitions. > > If you prefer these, I'd be glad to put them in and then indicate that > inverse multiplexing is the only form of composite link that is out of > scope for this document. Given that you stated that the use case document is the one that provides the definition/intro to IETF CLs, I'd just refer to that document and drop all references to G.800, including the Appendix, in this document. Also use the following definitions: Composite Link: As defined in [I-D.ietf-rtgwg-cl-use-cases]. Component Link: As defined in [I-D.ietf-rtgwg-cl-use-cases]. BTW I like your ideas WRT multipath, but I suspect that that ship has sailed. > >> - On a more detailed level, it seems that this document really just has >> MPLS (composite link) over MPLS (component links) use cases/solutions in >> mind. (Yes there is one line about other technology examples, but that >> is in just one spot and the MPLS is mentioned throughout.) If this is >> correct, then this should be made explicit in the early part of the >> document. If not, then the MPLS related assumptions/limitations need to >> removed. > > That is not true. The definition of component link should make this > clear. > > Component Link: A point-to-point physical link (including one or > more link layer) or a logical link that preserves ordering in > the steady state. > > If also gives examples which include Ethernet, PPP, SONET or OTN over > a physical link. It also states "A set of link layers supported over > pseudowire is a logical link that appears to the client to be a > physical link." > > In all cases it is MPLS over something, but not necissarily MPLS over > MPLS. I don't see a basis for the statement "it seems that this > document really just has MPLS (composite link) over MPLS (component > links) use cases/solutions in mind". > > Could you please point out any "MPLS related assumptions/limitations" > that are related to MPLS being used as a server layer rather than > using a link layer as a server layer. I don't see any. The existance > of section 4.2. Component Links Provided by Lower Layer Networks does > not state, nor does it in any way imply that only MPLS lower layers > are being considered. The one descriptive paragraph even gives > circuit switched or MPLS-TP as examples, further indicating that MPLS > over MPLS is not the sole focus. This section gives four requirements > related to passing information from the sever layer up to the client > layer that apply to the case of a server layer that is more complex > than a simple physical link plus link layer. Some of my comment was certainly biased by other drafts, including the framework draft which more clearly shows this. In terms of this document, I don't see how you expect arbitrary technologies used to provide component links to meet requirements FR#7 and FR#8. Do you really expect them to be met by non-MPLS (or perhaps GMPLS) controlled technologies? Did you think about making these objectives, or limited to MPLS/GMPLS controlled networks, rather than a requirement? Also, I read FR#11 as the labeling of the component not composite link, otherwise why mention "labeled traffic flow". > >> Minor Issues: >> >> - Abstract >> >> This whole section seems to be some early text that hasn't been >> revisited as the document progressed. I'd suggest throwing it out and >> just picking up (with slight modification) the Abstract from the >> cl-framework document which seems pretty good. > > Alia asked for an edit to the existing abstract. With the change that > Alia requested the abstract is now: > > There is often a need to provide large aggregates of bandwidth that > are best provided using parallel links between routers or carrying > traffic over multiple MPLS LSP. In core networks there is often no > alternative since the aggregate capacities of core networks today far > exceed the capacity of a single physical link or single packet > processing element. > > The presence of parallel links, with each link potentially comprised > of multiple layers has resulted in additional requirements. Certain > services may benefit from being restricted to a subset of the > component links or a specific component link, where component link > characteristics, such as latency, differ. Certain services require > that an LSP be treated as atomic and avoid reordering. Other > services will continue to require only that reordering not occur > within a microflow as is current practice. > > The point of the second paragraph is that some LSPs have specific > requirements (such as bounded delay) and some (most) do not. > Flexibility in accommodating both is the requirement. Composite Links > supporting non-homogenous Component Links is the basis for the > solution. That is why the framework mentions "group of homogenous or > non-homogenous links". > > The fact that Alia just commented on and we just changed the abstract > is indication that it has been read and is not just left over text > from ancient times that no longer applies. Could you please be more > specific about your objections to this text. It's also an indication that it needed some work. Given that you say the use-case is the source for IETF CL definition and that the abstract provides so much more information than the Intro, I'd suggest replacing with something really simple, perhaps along the lines of: This document provides a set of requirements for MPLS composite links. Composite link is a formalization of multipath techniques currently in use in IP and MPLS networks and a set of extensions to existing multipath techniques. > >> - 1. Introduction >> >> So where does one go to get an intro/definition of the IETF work on >> composite links? Is it G.800, the framework draft, the use cases draft, >> this draft? If the last, then this section needs a serious revision. If >> not, then this document should reference that definition and address the >> major issues identified above. > > I'd say "Use Cases". I'm in favor of de-emphasizing G.800. (Covered above). > > Please be clear. What major issues? "Major Issues" as identified in the RtgDir review template and above. > > Do you means the definitions of the well known terms CL and NPO? Yes. > If > so, how would you like them changed (see above). As covered above, reference use case document and ensure that the terms are appropriately defined there. > > Do you mean your belief that this document only applies to MPLS over > MPLS? In that case, you seem to be objecting to the intent of the > document even when your interpretation of the intent is contradicted > by many statements made in the document. If so, how can we make it > even more clear that a component link can also be a physical link, > circuit switched, pseudowire, besides mentioning that in the > definition of compnent link and in numerous other places. Covered above. > >> - 2 Assumptions >> I suspect this section will need to be updated to address the document >> scope comments above. > > AFAIK there is no change to the document scope. Given CL is defined in use cases, it seems more appropriate to me that Assumptions be presented as part of the definition. In other words, assuming the definition of CL is in the use case document, I think this it makes most sense for text to be in/moved to the use cases document. > >> - Definition of NPO. >> It seems to me that the authoritative definition of NPO is Y.1541 with >> Y.1540 providing additional detail. I think these documents would be a >> much more reasonable reference for NPO than the CL use case draft as is >> currently used. > > Would it be OK to add this text. > > Network operators have a contractual Service Level Agreement (SLA) > with customers for services that are comprised of numerical values > for performance measures, principally availability, latency, delay > variation. Additionally, network operators may have Service Level > Specification (SLS) that is for internal use by the operator. In > this document we use the term Network Performance Objective (NPO) > which may span multiple networks and may be looser than network > operator SLA or SLS objectives. The use of the term NPO in this > document is not entirely consistent with its definition in Y.1540 > and Y.1541 in that it is not restricted to UNI to UNI. There is no > intent to imply that a provider must base their SLA, SLS, or NPO on > Y.1540 and Y.1541. > > If we stay consistent with Y.1540 and Y.1541 then I think we have to > s/NPO/SLS or NPO/ and we did want a single term. Hereto, given the position that the definition of CL is in the use case document, I think the original reference makes sense. (Although I'd probably just say "Network Performance Objective (NPO): as defined in [I-D.ietf-rtgwg-cl-use-cases].") > >> - Many of the requirements refer to "meeting" or "not violating" NPO >> without given any specifics as to which NPO parameter/parameters is/are >> relevant. For example Class 5 QoS has unbounded upper bounds on Network >> performance parameter. Also NPO is defined UNI to UNI (end to end). I >> suggest adding a level of detail identifying the parameters and how the >> fixed values are to be applied relative to "meeting" or "not violating" NPO. > > Please see comments above regarding use of the term NPO. dito on the response ;-) > > We wanted one term. Diffserv uses SLA, even though there may be no > agreement with an outside party. If we can't use SLS because it is > specific to one provider only, can't use NPO because in Y.154[01] it > applies only to UNI to UNI, then we need another term. > > This reminds me of the host and router vs ES and IS arguments back in > the 1990s and the issue of using the term subnet to mean what it meant > in IETF since the early 1980s, ignoring what ITU had defined it to > mean. Eventually we dumped the use of Logical IP Subnetwork (LIS) as > we had to do in RFC1577 and could just say "subnet" again. > > Perhaps the best course of action is s/NPO/performance objective/ and > then define performance objective without any reference to Y.1540 and > Y.1541, but perhaps referring to SLA as defined in RFC2475. Independent of the terminology conundrum, you still have to figure out how to provide requirements that are meaningful in designing new protocols/mechanisms. I know this is a very difficult point, but without do so, it's hard to see how any solution can be evaluated against a requirement. BTW I really like the (relative terms) approach taken by DR#6 and DR#7 as they avoid any of the complications inherent in a specific-value performance parameter approach (such as seen in Table 1 of Y.1541) > >> - It might be useful to somehow separate data plane requirements from >> control plane requirements (Management/OAM requirements are already >> separated.) > > Too many of the requirements involve both the control plane and the > data plane. Where we are talking solely about physical links and > local decisions can be made, then no control plane signaling is > required. For some requirements, extensions to signaling would be > required for the MPLS over MPLS case. > > A good example is path symetry. In the framework, it is pointed out > that the most general case of path symetry on a per client LSP basis > (FR#22 and FR#23) is close to infeasible where there are multiple > client/server layer relationships. FR#22 and FR#23 works for classic > link bundling where each component link takes a single component > bidirectional path, but would require a lot of extension to work for > MPLS over MPLS over MPLS ad nauseum. > > Whether the solution is done in the data plane or control plane or > using a little contribution from each is a matter for the framework > document. The requirement is simply that an objective be met. > okay. Accepted & point closed. >> Nits: >> >> I have a bunch of more nit level comments, but I suspect much will >> change if/when the above are addressed, so I'm going to defer the >> majority of them until the above issues are resolved. > > I would like one more response from you before making any changes to > the document. > I agree, it makes sense to resolve the higher level comments before jumping on changes (and then on to nits.) >> - It looks like you are referencing an old version of g.800. The >> current is G.800 (02/2012). Also the G.800 "Cases" are identified in the >> draft not in G.800. > > I'm in favor of removing reference to G.800 except as a point of > clarification to say that they also define Composite Link and do so > somewhat differently. > I think I go a step further in my comments above and say drop it all together (and let it be covered in the CL-defining use case draft). >> Lou > > There is a lot of discussion above, mostly around definitions and > assumptions about document scope. > > In summary: > > 1. Remove the phrase "business problems" as described above. > Please let me know if the OLD/NEW change on this is sufficient. > > 2. In section 2, change "The services supported include ..." to > "Services which may be supported over MPLS based Composite Links > include ...". > > 3. Resovle issue of defintion of NPO. > > 3a. My preference is s/NPO/performance objective/. Remove > definition of NPO. Replace with: > > Performance Objective: Numerical values for performance > measures, principally availability, latency, and delay > variation. Performance objectives may be related to Service > Level Agreements (SLA) as defined in RFC2475 or may be > strictly internal. Performance objectives may span links, > edge-to-edge, or end-to-end. Performance objectives may span > one provider or may span multiple providers. > > 3b. Alternate is use the paragraph clarifying that we don't > quite mean NPO as defined in Y.154[01]. I'd rather just > get rid of the term NPO. > > 4. Replace definitions for "Composite Link" and "Component Link" > with those from draft-ietf-mpls-multipath-use and include > "Multipath" and "Inverse Multiplexing", plus "Classic Multipath" > as suggested above. > > 5. State that "Composite Link is also defined in ITU G.800 but the > usage here specifically excludes inverse multiplexing and is > more similar to multipath but with extensions to support groups > of homogeneous or non-homogeneous links. Remove all other > references to G.800. > > 6. Resolve wording on abstract. Please be more specific about your > objections to this text. > > 7. There is no change to document scope. Please indicate why you > think the document scope is not stated correctly. > > 8. Don't try to separate data plane vs control plane. In too many > cases that is more applicable to solution space than it is to > requirements. > > Please at least comment on only the summary before I make changes. > I think I responded to all the detail points. Let me know if I missed anything. Thanks again for the detailed response. Lou > Curtis > > > > _______________________________________________ rtgwg mailing list [email protected] https://www.ietf.org/mailman/listinfo/rtgwg
