Iftekhar,
Hello again and thanks for the follow up comments.
I've left your response intact but reformated. It seems we are in
agreement and there are some action items to be attended to. I've
listed the action items here and briefly commented inline.
#1 Some rewording is needed in Section 3 to clarify MPLS based
component vs physical link component, the latter inclucing the
link layer.
#2 Add a cross reference to CL framework in Section 4.1.
#3 Add cross reference to CL framework near FR#8 and FR#9 regarding
delay metric granularity and inclusion of queuing delay and the
potential impacts on performance, oscillations, and stability.
#4 Add a cross reference to CL Use Cases in Section 4.3 to point
toward the brief description and references that support the
statement "Many techniques have been developed to balance the
distribution of flows across component links that connect the
same pair of nodes" and "...via composite links, other
techniques have been developed."
#5 Briefly explain the "delay discontinuity" that occurs when load
balancing puts traffic on a new path and clarify what is meant
by "minimally disruptive".
#6 It may help in some cases to clarify that something must be
implemented but must, should, or may be configurable and
therefore may be optionally deployed. Doing so may require a
small change to many requirements as in most cases we have only
indicated where something must be implemented.
The rest of the email exchange seems to be clarifications. Please let
me know if I missed any of your suggestions in the action item list.
I will wait a week or longer for further comments from the WG and from
the other authors before making these changes and sending diffs to the
WG mailing list and after the WG has time to read the diffs I'll send
an updated draft.
Thanks and best regards,
Curtis
In message <d7d7ab44c06a2440b716f1f1f5e70ae534636...@sv-exdb-prod1.infinera.com>
Iftekhar Hussain writes:
> Curtis,
>
> Please see comments inline.
>
> Thanks for the detailed responses.
>
> Iftekhar
>
> > -----Original Message-----
> > From: Curtis Villamizar [mailto:[email protected]]
> > Sent: Thursday, April 19, 2012 7:10 PM
> > To: Iftekhar Hussain
> > Cc: [email protected]
> > Subject: Re: Ref: Composite link drafts update and request
> >
> >
> > Iftekhar,
> >
> > Responses inline. Some reformating of your email into plain text.
> >
> > Thanks for the interest and for the comments.
> >
> > Curtis
> >
> >
> > In message
> > <d7d7ab44c06a2440b716f1f1f5e70ae5345ca...@sv-exdb-prod1.infinera.com>
> > Iftekhar Hussain writes:
> >
> > > Hi,
> > >
> > > I have following clarification questions/comments about the
> > > https://tools.ietf.org/html/draft-ietf-rtgwg-cl-requirement-05
> > > document
> > >
> > > a) Section 3. "Examples of a physical link are: Lambda, Ethernet PHY,
> > > and OTN". Should "OTN" be "OTU"? Furthermore, can a "component
> > > link" be formed by set of Lambda?
> >
> > We are mixing physical and physical plus link layer in the examples.
> > The distinction is not important in this context but if you feel
> > strongly we could clarify.
> >
> > If we put ODU then others would complain that ODU is carried in OTU
> > and then we run into the issue that OTU is a framing on an OPU and
> > perhaps someone would argue that OTU is not a physical layer and that
> > even OPU is not truly a physical layer.
>
> [Iftekhar] Yes the main confusion was it was specifically referring to
> physical links. Your suggested change will help it clarify.
This is action item #1.
> > It might be best to change this to:
> >
> > Examples of a physical link (physical layer plus link layer) are: a
> > lambda with any link layer(s), Ethernet, OTN.
> >
> > Examples of multiple link layers include POS/SONET,
> > 10GbE/ODU2e/ODU4/OTU4/OPU4, etc. In the same way, WDM could be
> > considered an example of multiple physical layers (the individual
> > lambda multiplex using WDM). The problem is that any attempt to
> > strictly apply the antiquated ISO 7-layer model falls apart even is we
> > apply only the physical layer and link layer. For WDM we have the
> > physical layer subdivided into a layer zero (optical) and a layer one
> > (modulated signal aka single lambda). We have plenty of examples of
> > multiple link layer (layer two) protocols.
> >
> > The important distinction is [CL running over] a physical link plus
> > link layer [used as a component link] vs [CL running over] MPLS [used
> > as a component link]. Rewritten without the []: The important
> > distinction is a physical link plus link layer vs MPLS.
> >
> > > b) Section 4.1. In the context of "FR#1" from end-to-end perspective,
> > > should there be some sort of network scale related requirement(s)?
> > > For example, max number of pair of nodes connected via composite
> > > links traversed etc. If this aspect is covered somewhere a
> > > reference should be added.
> >
> > End-to-end in any MPLS context means edge-to-edge (and is not
> > mentioned at all in FR#1 or elsewhere). Existing scaling techniques
> > used in MPLS networks still apply. Scaling is the motivation for DR#5
> > (IGP extensions disallowed in multiple domain case). Scaling is
> > directly addressed in DR#6 (fast convergence) and DR#7 (fast worst
> > case failure convergence).
> >
> > Scalability and stability are covered in the framework document
> > (draft-so-yong-rtgwg-cl-framework). For example in "3.1. Scalability
> > Motivations" and "3.5. Avoiding Route Oscillation". Possible need
> > for further work is suggested in "7.2.11. Performance, Scalability,
> > and Stability" in the framework document.
>
> [Iftekhar] Okay. I will take a look at the "
> draft-so-yong-rtgwg-cl-framework". You might consider adding a
> reference to the framework document.
This is action item #2.
> > > c) Section 4.2, "Communication of other performance parameters (e.g.,
> > > delay variation) is desirable" seems to contradict "FR#17" in
> > > section 4.3.
> >
> > The statement is:
> >
> > [...] Communication of the latency performance
> > parameter is a very important requirement. Communication of other
> > performance parameters (e.g., delay variation) is desirable.
> >
> > If jitter was described as "undesireable" then it might be a
> > contradiction. The point is delay is very important and delay
> > variation would be nice (maybe not IMHO). What is important (ie:
> > normative) is what the implementation MUST support, and what the
> > operator MAY enable or disable. What is unimportant is subjective
> > verbiage about relative importance among parameters. The operator
> > will decide what is important by using or not using a feature.
> >
> > FR#17 says:
> >
> > FR#17 The solution SHALL provide a means to indicate that a traffic
> > flow shall select a component link with a maximum acceptable
> > delay variation value as specified by protocol.
> >
> > There are two differences between the statement in Section 4.2 and the
> > statement in FR#17. First "communication of other performance
> > parameters" and "provide a means to indicate that a traffic flow shall
> > select a component link with a maximum acceptable delay variation
> > value" are two different things; the first is a IGP parameter (link
> > delay or link delay variation), the second is an RSVP parameter
> > (maximum allowable delay variation for a given LSP). Second, the
> > wording in Section 4.2 is informative and the wording in FR#17 is
> > normative.
>
> [Iftekhar] Okay. I interpreted jitter as "desirable" meaning it is an
> objective i.e., it is not required. But then saw the requirement FR#17
> which I read as a requirement to support delay variation. For example
> in FR#17 I was expecting "MAY" or "SHOULD" rather than "MUST".
See later comment referring to item #6.
> > > d) Section 4.2, FR#8" appears to use "latency" in the end-to-end
> > > sense. While the next requirement "FR#9" appears to refer to this
> > > term only between two nodes. I think, it would helpful to clarify
> > > the usage of term "latency" e.g., what component of a node transit
> > > delay it excludes or includes. Or add a reference to a document if
> > > this is covered in some other document.
> >
> > BTW- I prefer delay over latency, but the term latency is used. The
> > two mean the same thing. Neither word by itself does not indicate any
> > one type of delay (fabric delay, transmit delay, queuing delay,
> > geographic delay). Neither word by itself implies intra-node delay,
> > or delay between adjacent nodes, or end-to-end delay.
> >
> > FR#8 applies to any measurement of delay, whether it is a measurement
> > over a single physical link or over a component link carried over a
> > multihop MPLS LSP. Also in FR#9 there is no indication that the nodes
> > are adjacent, and the phrase "when the path between these nodes
> > contains one or more pairs of nodes connected via a composite link"
> > clearly indicates that multihop paths are being considered as well as
> > delay between adjacent nodes.
> >
> > There are some hints as to which delays the operators may consider
> > important in this context. FR#8 states that "The precision of latency
> > reporting SHOULD be at least 10% of the one way latencies for latency
> > of 1 ms or more." This excludes most equipment latencies which can be
> > expected to be well under 100 usec (10% of 1 msec).
> >
> > The intent is to measure the predominant latency in most (well run)
> > service provider networks, which is geographic delay on the order of
> > msec or tens of msec and in some cases over 100 msec (intercontinental
> > traffic). In a congested network (possibly not so well run network,
> > or maybe a well run network with a substantial outage) queuing delay
> > can be quite large. In a congested network measuring the queuing
> > delay for the low priority traffic (the only traffic likely to be
> > experiencing congestion, even in an outage) is more likely to add to
> > problems by adding routing instability.
> >
> > The only real question in FR#8 and FR#9 is whether queuing delay is
> > counted in the latency figure. The argument for including queuing
> > delay is that it reflects the delay experienced by applications. The
> > argument against including queuing delay is that it if used in routing
> > decisions it can result in routing instability.
> >
> > There are quite a few documents that deal with this tradeoff. For
> > example, in MPLS-TP it is noted that delay measurements should be made
> > using the highest priority COS value (which in effect very nearly
> > eliminates any queuing delay from the measurement).
> >
> >
> > For the requirements document, this tradeoff is not discussed, however
> > the stability and convergence requirements must be considered when
> > coming up with a solution. Solutions should be described in the
> > framework document and not in the requirments document.
> >
> > In the framework document (draft-so-yong-rtgwg-cl-framework) there is
> > discussion of this tradeoff in "3.5. Avoiding Route Oscillation" and
> > elsewhere. The framework document is a better place for discussion of
> > requirements tradeoffs.
>
> [Iftekhar] Okay, will take a look at the framework document.
A cross reference could be added. This is action item #3.
> > > e) General comment section 4.3. "Many techniques have been developed
> > > to balance the distribution of flows across component links that
> > > connect the same pair of nodes" and "...via composite links, other
> > > techniques have been developed." It would good to add a
> > > reference(s) for each.
> >
> > There are a few RFCs on the topic. Inverse multiplexing, various
> > forms of ECMP, and Ethernet Link Aggregation are all well known. The
> > statement would certainly not be controversial without references.
> >
> > If you are looking for further information on this, the Use Cases
> > draft (draft-symmvo-rtgwg-cl-use-cases) has references. "Appendix B.
> > Existing Multipath Standards and Techniques" was moved from the
> > requirements document to the Use Cases draft.
> >
> > We could add a reference to the use cases draft in the requirements
> > draft if you think that would help.
>
> [Iftekhar] I believe the requirement document would read better and
> easier to follow if the references are added.
This is action item #4.
> > > f) Section 4.3, "FR#12" is about latency while the example "...a user
> > > experience objective (e.g. jitter buffer under/overrun)" appears to
> > > refer to delay variation. I think, the example should be consistent
> > > with the requirement.
> >
> > The requirement in FR#12 is in the first paragraph.
> >
> > FR#12 When a traffic flow is moved from one component link to
> > another in the same composite link between a set of nodes (or
> > sites), it MUST be done so in a minimally disruptive manner.
> >
> > The rest is discussion.
> >
> > When a flow is moved from a current link to a target link with
> > different latency, reordering can occur if the target link
> > latency is less than that of the current or clumping can occur
> > if target link latency is greater than that of the current.
> > Therefore, some flows (e.g., timing distribution, PW circuit
> > emulation) are quite sensitive to these effects, which may be
> > specified in an NPO or are needed to meet a user experience
> > objective (e.g. jitter buffer under/overrun).
> >
> > The discussion exists primarily for the reader who might otherwise not
> > realize why there would be a disruption at all (and therefore might
> > not understand why we have to say "minimally disruptive" instead of
> > "completely non-disruptive"). Perhaps the entire paragraph needs to
> > start with "For example, [...]".
> >
> > In any case it is not inconsistent. When delay changes on the current
> > big-bad-Internet (such as in VOIP), the change is often absorbed by a
> > jitter buffer as long as the change is small. If a delay change of
> > many milliseconds occurs, then it is unlikely that the typical
> > relatively small jitter buffer can hide the delay change.
>
> [Iftekhar] Thanks for detailed response. To me then the requirement
> then is about change in latency (i.e., delay variation).
Delay variation or jitter is a continuous variation in delay. It is
somewhat like white noise added to the otherwise constant delay,
though queuing delay when congestion occurs adds less random patterns
of delay change, often with a period of multiples of RTT. When a
fault occurs or is reparied or load balancing puts traffic on a new
path, there is a sudden one-time change in delay. This is referred to
as a "delay discontinuity" and its magnitude is often far greater than
the delay variation, including delay variation that results from
queuing delay under congestion. An occasional delay discontinuity is
considered minimally disruptive.
It would not hurt to briefly explain this effect and introduce the
term "delay discontinuity". This would clarify what we mean by
"minimally disruptive", though this may be one example of a minimally
disruptive behavior. I have listed this as action item #5.
> > > g) Section 4.3, "FR#17". It is not clear why the delay variation is
> > > considered in this context. Reading this requirement, one gets the
> > > impression as if the composite link delay has two components (delay
> > > and delay variation). I had understood that delay variation
> > > (contributed through the nodal switching/queuing delays) is not
> > > considered in this document.
> >
> > FR#16 is about delay. FR#17 is about delay variation. The wording
> > "The solution SHALL provide a means" doesn't require the operator to
> > use that feature. Therefore it is better to keep the two in separate
> > requirements as it is more clear that one could be enabled and the
> > other disabled.
>
> [Iftekhar] I think it would be clear if somehow the meaning of the
> above sentence "i.e., doesn't require the operator...." is included in
> the text.
This is item #6. See text at the top of the email message.
> > > h) DR#6, a minor typo "Solution" should be "solution"?
> >
> > Thanks for pointing this out.
> >
> > > Thanks,
> > > Iftekhar
> > >
> > > > As was discussed in yesterday's rtgwg meeting, there are three
> > > > composite link drafts:
> > > >
> > > > https://tools.ietf.org/html/draft-ietf-rtgwg-cl-requirement-05
> > > >
> > > > https://tools.ietf.org/html/draft-so-yong-rtgwg-cl-framework-05
> > > >
> > > > https://tools.ietf.org/html/draft-symmvo-rtgwg-cl-use-cases-00
> > > >
> > > > You can read the slides I presented on these drafts at
> > > > https://tools.ietf.org/agenda/83/slides/slides-83-rtgwg-3.pdf (it's
> > > > a quick read!).
> > > >
> > > > As Alia said, you are all requested to read and comment on these
> > > > drafts. In about a month's time, we will be requesting WG adoption
> > > > of draft-so and draft-symmvo, and it'll be great to have an informed
> > > > discussion at that time.
> > > >
> > > > Thanks much,
> > > > Andy
_______________________________________________
rtgwg mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/rtgwg
