Iftekhar, I replied to the wrong message. This one came after the one I replied to. I just looked this over and it appears that I didn't miss anything in this reply.
Regarding your last point: [...] It would not be difficult to put in the reverse cross references (functional group refers to protocol change or functionality supporting it). [Iftekhar] OK. That would be great. One thing I noticed was that some of the sections in this document are referring to specific requirements that are being addressed while others don't Actually it is not difficult in principle to provide cross references but it is somewhat time consuming. It also turns out that it isn't very useful. The reason is that documents covering many topics must consider certain groups of requirements such as backward compatibility and general network management. I did the exercise briefly, but didn't expand the text yet. I'll post diffs if it seems to add more clarity than bloat. Regarding "One thing I noticed was that some of the sections in this document are referring to specific requirements that are being addressed while others don't": A number of requirements are cited because they highly problematic or have non-obvious implications. Many such citations are within the "New Challenges" section in cases were a particular requirement is problematic or in the "Existing Mechanisms" or "Mechanisms Proposed in Other Documents" because existing or proposed mechanisms address some requirements or because the mechanisms are incomplete in some way, failing to address some requirements. There are citations in "Required Document Coverage" to indicate specific requirements or sets of requirements that are going to need a document to cover the approach taken to address them. This is the full set of requirements cited before "7.1. Brief Review of Requirements". FR#10 and FR#12 are cited as requiring the existing soft preemption [RFC5712]. The delay and jitter requirements are listed to emphasize that there are quite a few or them and not all in the same section followed by the sentence "Requirement FR#17 is particularly problematic, calling for constraints on jitter" and then discussion. FR#18 and FR#19 are cited because they indicate local control which FR#21 gives control to the ingress. FR#21 is again cited because it specifies symetrical traffic, a requirement which may not be acheivable with very large LSP. This is the full set of requirements cited after "7.1. Brief Review of Requirements". Most of these citations are in "7.2. Required Document Coverage". The CL framework document would be very long if it attempted to fully address all of the issues. The following are sentences pulled from the text. FR#11 explicitly calls for dynamic load balancing. IGP-TE and RSVP-TE extensions are needed to support frequency of load balancing rearrangement called for in FR#13, and FR#15-FR#17. Lower layer to upper layer communication called for in FR#7 and FR#20. The behavior of hash methods used in classic multipath needs to be described in terms of FR#12 which calls for minimally disruptive load adjustments. Protocol extensions are needed to support dynamic load balance as called for to meet FR#22 (path symmetry) and to meet FR#11 (dynamic placement of flows). Extending LDP is called for in DR#2. DR#5 calls for Composite Link to span multiple network topologies. There is one citation in "7.3. Open Issues Regarding Requirements". The following sentence appears in that section: "We may need a performance section in this document to specifically address #DR6 (fast convergence), and #DR7 (fast worst case failure convergence), though we do already have scalability discussion." The paragraph is asking whether we need a document to cover this or whether the coverage in the framework is sufficient. If we have a separate document, then we can pull some text out of the CL framework. Curtis In message <d7d7ab44c06a2440b716f1f1f5e70ae534645...@sv-exdb-prod2.infinera.com> Iftekhar Hussain writes: Curtis, Thank you for the detailed responses. Please see in-line. Best regards, Iftekhar -----Original Message----- From: Curtis Villamizar [mailto:[email protected]] Sent: Sunday, May 06, 2012 11:14 AM To: Iftekhar Hussain Cc: [email protected] Subject: Re: draft-so-yong-rtgwg-cl-framework Iftekhar, Thank you for the detailed comments. Responses to individual comments/suggestions are inline. This is a long email message, so if I missed anything, please point out what I missed. Please let me know if you are in general agreement with the responses and if so I'll make specific proposals for replacement text or propose more detailed action items. Best Regards, Curtis In message <d7d7ab44c06a2440b716f1f1f5e70ae534638...@sv-exdb-prod1.infinera.com> Iftekhar Hussain writes: > Dear Authors, > > Please find below some comments on the > http://tools.ietf.org/html/draft-so-yong-rtgwg-cl-framework-05. > > > ---------------------------- > > 2.1. Flow Identification > > "Operator may have other objectives such as ...composite link energy > saving, and etc. These new requirements are described in > [I-D.ietf-rtgwg-cl-requirement]" > > Comment: > > I don't recall any energy saving related requirement (or discussion) > in the [I-D.ietf-rtgwg-cl-requirement. Suggest removing the text > "energy saving etc." You are entirely correct that there are no energy savings mentioned in the requirements document. This wording came from one of the other authors but I think I understand the intent. There is a daily cycle (and less so a weekly cycle) to traffic levels. During the low traffic hours (generally late at night for local hours of night) traffic can be rebalanced to reduce the number of active component links. Where it is possible to depower interfaces, this could result in energy savings. I'm OK with removing this example, though it is a very good example and an goal that we should be pursuing. Another option is to add a "MAY" in the requirements docuemt somewhere that indicates that "Load balancing MAY be used during sustained low traffic periods to reduce the number of active component links for the purpose of power reduction". [Iftekhar] OK. The latter option seems reasonable. I'll wait for further comments on this before making a suggestion on how we will proceed. > 2.2. Composite Link in Control Plane > > "LDP follows the IGP, therefore failure to forward on the IGP path > will often result in loss of connectivity if the IGP adjacency is not > withdrawn when an LDP FEC is refused. This is a pathologic case that > can occur if LDP is carried natively and there is a high volume of LDP > traffic. This situation can be avoided by carrying LDP within RSVP-TE > LSP." > > > Comment: > > Is it the loss of connectivity referring to LDP control plane > connectivity only or LDP signaled LSP data plane or both? The bottom line is that admission control cannot be applied to LDP without loss of connectivity. Complete loss of connectivity for a subset of traffic, no matter how low its priority, is unacceptable. LDP does not support TE. There are two options. Either don't carry enough LDP traffic such that this situation can occur (this is the case when a small subset of traffic is high priority traffic carried within LDP, such as VOIP or enterprise VPN traffic mixed with a much larger volume of Internet traffic). The alternative is to carry the LDP traffic within RSVP-TE LSP when enough LDP traffic is aggregated such that TE is needed. The goal was to refer to this problem without going into an excessively long explanation. If you think it is too brief and therefore is unclear, then I'll reword it. [Iftekhar] OK. I believe a brief rewording to help clarify this point would be useful. > Comment: > > "Composite link capacity is aggregated capacity and MAY be larger than > individual component link capacity." Before the MAY insert "LSP capacity" and the sentence makes more sense. I'll reread this and see if that is what was intended. > Composite link aggregate capacity should always be larger than > individual component link capacity. Did I misunderstand? The statement as written is not useful. The sum of positive quantities is always greater than the quantities being summed. Thanks for pointing out this statement. I'll look at the context and figure out the intent. [Iftekhar] Okay. That will be great. Thanks. > > "...1. If no other information is available the largest microflow is > bound by one of the following:" > > Suggested text: > > If no other information is available the largest microflow is bound > (i.e., signaling extensions don't indicate a bound) by one of the > following: I'll reword this. The other information could be signaled or configured. [Iftekhar] OK. > Comment: > > Section 2.2 provides architectural guidelines e.g., " ...Available > capacity in other component links MUST be used to carry impacted > traffic. The available bandwidth after failure MUST be advertised > immediately to avoid looped crankback" and provides illustrative > examples e.g.,"... no microflow larger than 10 Gb/s will be present on > the RSVP-TE LSP that aggregate traffic across the core, even if the > core interfaces are 100 Gb/s interfaces." Be careful not to take the last sentence out of the context of the example (where no interface feeding the core is greater than 10 Gb/s). > In contrast, section 2.1 appears to be little bit too vague e.g., " > ... technique of grouping flows, such as hashing on the flow > identification criteria, becomes essential to reduce the stored state, > and is an essential scaling technique. Other means of grouping flows > may be possible" > > Suggestion: > > Add some examples which flow identification scheme to use for > composite links or add a reference to section 4.2 which discusses flow > identification trade-offs. The intent in Section 2.1 is to make three points. First, tracking every flow is not scalable. Second, IP src/dst address hashing has proven (in over two decades of experience) to be an excellent way of identifying groups of flows (for reasons briefly listed). Third, if you find a better way to identify groups of flows, then use it. We don't want to require the use of IP address hashing, but wanted to strongly encourage its use, given its long history of successful deployment. I'll look at rewording the section. [Iftekhar] Okay. I believe that will be helpful to the reader. > 4.1.4. Requirements for Contained LSP > > Comment: > > Add reference to specific relevant to requirements in > I-D.ietf-rtgwg-cl-requirement] similar to section 4.1.2 and 4.1.3. OK. The sentence "[I-D.ietf-rtgwg-cl-requirement] calls for new LSP constraints." needs followup indicating which requirements are being referred to. [Iftekhar] OK. Thanks. > 4.2. Data Plane Challenges > > Comment: > > Minor typo: "very course..." should be "very coarse..." Of coarse. How could I miss that? :-) I searched for other uses of the word "course" and found quite a few that should be "coarse". Thanks for the catch. [Iftekhar] OK > Comment: > > "In practice using the MPLS label stack alone has proven too course > to acheive a reasonably good load balance, due to bin-packing issues > and discrpencies between signaled bandwidth and actual traffic loads > on LSP." > > Suggest adding a reference to an IETF standard or best practices > document that discusses these issues. OK, if I can find something. This has been discussed on and off on the mailing lists for about a decade. There may be something in the original link bundling RFC or elsewhere. [Iftekhar] OK > Comment: > > Sections 4.2, 2.1, 2.3, appear to be somewhat redundant. Suggest > trimming section 2.1 and absorbing that information in section 4.2. Thanks for pointing this out. I'll reread and try to reduce redundancy, using cross references where appropriate rather than repeating a point. [Iftekhar] OK > 3. Architecture Tradeoffs > > Comment: > > "Composite Link is applicable to large networks, and therefore > scalability must be a major consideration." > > What is a typical definition of a large network? Suggestion: Add an > example (or a forward reference to section 3.3.1 which has an example) Any network that one of your customers wants to build would be a good example of large. :-) [But not a good definition of large.] By "large" we mean the type of networks that service providers or large content providers are building today. In the future we may find that enterprises are building large enough private networks to have certain scaling problems that had previously only been experienced by service providers and content providers in the past. A good example of that in the past is overuse of bridging. Some of the NSFNET regional networks in the late 1980s used bridging but experienced severe scaling issues very quickly. It wasn't until the early to mid 1990s that large enterprises began having similar problems. The point of the phrase "Composite Link is applicable to large networks" is that if you need parallel links because the single link capacity je jour is too small, then your network is large relative to most other networks. Considering the magnitude of "large" where 10Gb/s links are too small, the phrase "and therefore scalability must be a major consideration" may seem almost rhetorical to some with deployment experience with IP networks. The reason that the statement here is brief is that there are a number of IETF base documents dating back to the mid-1990s on the importance of scalability. These realization came from deployment experience. Despite this someone occasionally makes a naive comment about processors getting faster and memory getting larger, ignoring the fact that comparable grouth with order NlogN or N-squared growth in computation or memory requirements far outpaces Moore's Law improvements. I think at least one somewhat ancient RFC indicatea that scalability must be considered in every document in the IETF routing area. If I find it I'll cite it. It will probably have Brian Carpenter or Christian Huitman on the author list, making it easier to find. Maybe Fred Baker. I think it was an IAB or IESG RFC. [Iftekhar] OK. I understand the difficulty. > 3.1. Scalability Motivations > > Comment: > > "....a large routing change to be accomplished more quickly," > > What is a typical definition of a large routing change? Suggestion: > Add an example. Not entirely serious: Hurricane Katrina resulted in a number of "large routing changes". There was the collaspse of the Raratan River bridge in NJ taking out fibers on both sides of the bridge (thought to be redundant), an earthquake east of San Diego taking out three of four fiber paths, etc. This is again a relative term that has been used for quite some time. A customer circuit going down or a new customer coming online results in a tiny routing change. A metro fault may be handled entirely in the metro and have no effect at all on routing elsewhere in the provider network or global network. Major fiber outages generally result in large routing changes. In an MPLS context, a large routing change is one that affects a large number of LSP. A classic example is where one hop along one of two or three east-west paths across the continental US goes down (or comes up, though this can be made far less disruptive). Many LSP traverse the small number of US east-west paths and terminate in a large number of medium to large sized cities along either coast. Again, I'll look at clarifying, but in less words than this response. [Iftekhar] OK. I will leave to your discretion. May be have a "typical" today large network numbers? > 7.2.5. Dynamic Multipath Balance > > Comment: > > "...uses a course granularity, the adjustments would have to be > equally course, in the worst case moving entire LSP" > > Minor typo "course" should be "coarse". s/course/coarse/ with selective replacement. [Iftekhar] OK. :) > 7. Required Protocol Extensions and Mechanisms > > Comment/suggestion: > > Organizing section 7 as a matrix containing something like: > Requirement#, Existing Mechanisms, Gaps, Ongoing/new extensions..., > might be more clearer. AT a minimum, add a traceability to each > requirement in [I-D.ietf-rtgwg-cl-requirement] and the section number > in framework document that addresses each requirement (or set of > requirement). Section 7.2 groups the requirements but there is no where that lists every requirement in numeric order and points to where in the grouping it falls. It should be easy enough to add that. At the moment, potential issues with the requirements or in this document are listed in Section 7.3. This is a different approach, listing the omissions rather than cross referencing. Ideally we address all of the issues and drop this section. Section 7.4 lists the requirements by protocol or functionality affected. This as stated in the first paragraph is for the benefit of implementors. Subsections of 7.4 refers back to the groupings in subsections of 7.2. It would not be difficult to put in the reverse cross references (functional group refers to protocol change or functionality supporting it). [Iftekhar] OK. That would be great. One thing I noticed was that some of the sections in this document are referring to specific requirements that are being addressed while others don't > Is there a minimal set of requirements which must be met to form a > deployable (useful) composite link based solution? That's a great question. "Enough to be useful" is interpreted differently by different network operators deploying a network. For an equipment vendor the best approach is to plan to implement everything over time under the assumption that every feature will appear in a check box in someone's RFI/RFP/RFQ eventually, but ask current key customers which features to prioritize. That process may yield a different answer for different equipment vendors, depending on who their current key customers are. BTW- Our friends at Verizon made sure that almost every requirement is a MUST, so ask them to identify the requirements that they are not really serious about. Wear protective fireproof undergarments. :-) [Iftekhar] OK, :) > ----------------------- > > Regards, > Iftekhar _______________________________________________ rtgwg mailing list [email protected] https://www.ietf.org/mailman/listinfo/rtgwg
