Hi Deepak, very interesting and helpful to learn from your experience with latency/jitter measurement. But I think that there are different levels of details:
- NVO3 packet format; - NVO3 OAM requirements and gap analysis. For the former, I think OAM flag and two bit long marking field that can be used in passive performance measurements are sufficient to support both active and passive FM and PM. As far as latency/jitter measurement, we can look into re-using work of IPPM and MPLS WGs as much as possible. OWAMP/TWAMP, on one hand, and RFC 6374, on another, should be mapped to NVO3 OAM requirements, IMO. True, we may find that support of both one-way and two-way PM methods required. I'd just note, that one-way latency/jitter measurement does require clock synchronization. Two-way can measure round-trip with or without good clock synchronization between sender and reflector. If the former case, forward and reverse latency/jitter as well as two-way can be measured and calculated. If the latter - round trip only. Personally, I find one-sided two-way PM more useful. As far as timestamp format, approach of RFC 6374 seems as most flexible as it allows use of NTP and truncated 1588v2 timestamps. Regards, Greg On Sun, Nov 16, 2014 at 8:44 AM, Deepak Kumar (dekumar) <[email protected]> wrote: > Hi, > > Please see inline +++DK: > > On 11/14/14 11:09 AM, "Jon Hudson" <[email protected]> wrote: > > > > >One comment in line.... > > > >> On Nov 13, 2014, at 11:47 PM, Vero Zheng <[email protected]> wrote: > >> > >> Hi Tom, > >> > >> Please see in-line. > >> > >> BR, Vero > >> > >>> -----Original Message----- > >>> From: nvo3 [mailto:[email protected]] On Behalf Of Tom Herbert > >>> Sent: Friday, November 14, 2014 4:27 PM > >>> To: Mach Chen > >>> Cc: Greg Mirsky; Haoweiguo; Marc Binderberger; Larry Kreeger; > >>>[email protected] > >>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane requirements for > >>> OAM > >>> > >>> On Wed, Nov 12, 2014 at 5:13 PM, Mach Chen <[email protected]> > >>> wrote: > >>>> Hi Tom, > >>>> > >>>>> -----Original Message----- > >>>>> From: Tom Herbert [mailto:[email protected]] > >>>>> Sent: Thursday, November 13, 2014 3:11 AM > >>>>> To: Marc Binderberger > >>>>> Cc: Mach Chen; Greg Mirsky; Haoweiguo; [email protected]; Larry Kreeger > >>>>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane requirements for > >>>>> OAM > >>>>> > >>>>> On Wed, Nov 12, 2014 at 2:11 AM, Marc Binderberger <[email protected]> > >>> wrote: > >>>>>> Hello Mach, > >>>>>> > >>>>>> so for delay measurement you use the color flag to mark a single > >>>>>> packet, which helps the receiver to pick the right packet? And > >>>>>> repeat this every time period T ? > >>>>>> > >>>>>> ...000100000010000001000... > >>>>> Is there there a draft or description of how this algorithm would > >>>>> work? Seems like there would need to be quite a bot of > >>>>> synchronization needed between end points (synchronized clocks, > >>>>> provisions to correlate measurements correctly with lost packets, > >>>>> replicated packets, etc.). Also, what is envisioned for range for > >>>>>the period? > >>>> > >>>> Here is a reference > >>> > >>> > https://datatracker.ietf.org/doc/draft-chen-ippm-coloring-based-ipfpm-fr > >>>amew > >>> ork/. > >>> > >>> Thanks for the pointer. Regarding the need for synchronized clocks to > >>>measure > >>> delay, I consulted our local NTP expert. The host clock jitter we > >>>currently see in > >>> our network is currently usually greater than one-way packet delay (in > >>>some > >>> cases much greater), so in his words: > >>> "measuring one-way packet delays using host clocks is a lost cause". > >>> Please take this as just one data point! > > > ><Jon> Thank you. As someone who has managed NTP more times and for more > >years than I care to admit, this is a very good datapoint to consider. > >NTP helps many understand that time is relative. > > +++DK: As per our experience in carrier Ethernet we supported one way > delay and never found NTP useful even for our lab networks (I am referring > software based NTP NTPv3). > As mentioned below IEEE 1588v2 will vary based on equipment and operator > networks but in our testing we found it very precise if properly deployed. > IEEE 1588v2 is very precise if phy based timestamping is used. Even > timestamping at NP level provided great results for one way delay. > > If we want to accurately measure two way delay we need 4 timestamp total > on receiver of frame (this is to avoid processing time that's taken for > reply by software as hardware can put timestamp at lower layer without > doing delay and jitter calculation). > For one way delay we will require 2 timestamp, so lower layer hardware can > timestamp before packet is punted to software. > > As mentioned below I agree 8 byte IEEE 1588 timestamp is required. > > We should also look for Synthetic OAM applicability for performance ('O' > bit can be overloaded to do both Fault and performance if OAM is defined > with different oam Subtype for Delay and Loss frames and it will not be > too deep hardware inspection) as that give large flexibility > (synthetic/real loss measurement, Availability/unavailability, on-demand > and pro-active performance) and can be run on all flows of ECMP. > > Thanks, > Deepak > > > > > >> > >> [Vero] Thanks for this. What about the current experience with 1588v2 > >>then? > >>> > >>>> Yes, it does need some synchronization. As for the range, it depends > >>>>on two > >>> factors, one is the implementation limitation, the other the > >>>requirement of the > >>> operators. In the above reference, the suggested periods are 1s, 10s, > >>>1min, > >>> 10min and 1h. > >>> I think if we were implementing delay measurement in GUE, I would > >>>advocate > >>> add a 64 bit optional field for timestamp, probably containing source > >>>time > >>> stamp, and echoed timestamp for a flow (usec resolution and similar in > >>>design > >>> TCP timestamp option). This easily gives a precise RTT, and if clocks > >>>are > >>> precisely synchronized then one way latency could be calculated also. > >> [Vero] If the source timestamp could be carried, it could also be used > >>for packet loss calculation/correlation. > >> > >>> Thanks, > >>> Tom > >>> > >>>> Best regards, > >>>> Mach > >>>>> > >>>>> Thanks, > >>>>> Tom > >>>>> > >>>>>> > >>>>>> One question I still have is: why is the measurement done in the NVE > >>> header? > >>>>>> The outer header is IP/IPv6, so couldn't we use the coloring for > >>>>>> the > >>>>>> IP/IPv6 header, assuming this is defined? > >>>>>> > >>>>>> > >>>>>> Thanks & Regards, > >>>>>> Marc > >>>>>> > >>>>>> > >>>>>> > >>>>>>> On Wed, 12 Nov 2014 09:34:52 +0000, Mach Chen wrote: > >>>>>>> Hi Tom, > >>>>>>> > >>>>>>>> -----Original Message----- > >>>>>>>> From: Tom Herbert [mailto:[email protected]] > >>>>>>>> Sent: Wednesday, November 12, 2014 5:06 PM > >>>>>>>> To: Mach Chen > >>>>>>>> Cc: Greg Mirsky; Haoweiguo; [email protected]; Larry Kreeger > >>>>>>>> (kreeger) > >>>>>>>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane requirements > >>>>>>>> for OAM > >>>>>>>> > >>>>>>>> On Wed, Nov 12, 2014 at 12:55 AM, Mach Chen > >>>>>>>> <[email protected]> > >>>>>>>> wrote: > >>>>>>>>> Hi Greg and all, > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Single bit is not sufficient if someone wants to perform loss > >>>>>>>>> and delay measurement simultaneously, then two bits needed. > >>>>>>>> Is that necessary? Can they share the same time quantum (as well > >>>>>>>> as other metrics maybe to be added later)? In all the protocols > >>>>>>>> mentioned, the reserved bits are a somewhat precious resource. > >>>>>>> > >>>>>>> Yes, it's necessary if there is ECMP. > >>>>>>> > >>>>>>> Given one bit is used for both loss and delay measurement, for > >>>>>>> loss measurement, it periodically set and clear the marking bit, a > >>>>>>> flow is divided into consecutive blocks, and then the counting and > >>>>>>> calculating are based on each block. This is fine for loss > >>>>>>>measurement. > >>>>>>> > >>>>>>> For delay measurement, it has to make sure the timestamps > >>>>>>> (collected at sender and receiver) are for the same packet. > >>>>>>> Presumably, the time when changing the marking bit is right time > >>>>>>>to get > >>> the timestamps. > >>>>>>> Since there is ECMP, the first packet of a block at the sender may > >>>>>>> probably different from the first packet at the receiver, thus it > >>>>>>> will get the mismatched timestamps to calculate the delay. > >>>>>>> > >>>>>>> Best regards, > >>>>>>> Mach > >>>>>>>> > >>>>>>>> Tom > >>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Best regards, > >>>>>>>>> > >>>>>>>>> Mach > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> From: nvo3 [mailto:[email protected]] On Behalf Of Greg > >>>>>>>>> Mirsky > >>>>>>>>> Sent: Wednesday, November 12, 2014 8:05 AM > >>>>>>>>> To: Haoweiguo > >>>>>>>>> Cc: [email protected]; Larry Kreeger (kreeger) > >>>>>>>>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane > >>> requirements > >>>>>>>>> for OAM > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Dear All, > >>>>>>>>> agree with Weiguo, single bit flag in fixed position would be > >>>>>>>>> sufficient and HW-friendly. > >>>>>>>>> > >>>>>>>>> Regards, > >>>>>>>>> > >>>>>>>>> Greg > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Tue, Nov 11, 2014 at 3:51 PM, Haoweiguo > >>>>>>>>> <[email protected]> > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> Hi Larry, > >>>>>>>>> > >>>>>>>>> For marking purpose, i think one bit maybe OK, fixed fields in > >>>>>>>>> NVO3 header is precious. I would like it is set in fixed field, > >>>>>>>>> rather than in option field. Because chipset normally can't > >>>>>>>>> process optional field, it is hard to realize in-band > >>>>>>>>> performance measurement if using optional > >>>>>>>> field for marking. > >>>>>>>>> For other real time congestion control function, maybe more bits > >>>>>>>>> are needed. > >>>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> > >>>>>>>>> weiguo > >>>>>>>>> > >>>>>>>>> ________________________________ > >>>>>>>>> > >>>>>>>>> 发件人: Larry Kreeger (kreeger) [[email protected]] > >>>>>>>>> 发送时间: 2014年11月12日 4:33 > >>>>>>>>> 收件人: Haoweiguo; Greg Mirsky > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> 抄送: [email protected] > >>>>>>>>> 主题: Re: [nvo3] Comments on NVO3 data plane requirements for > >>> OAM > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Hi Weiguo, > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> What do you envision this marking looking like? e.g. is it just > >>>>>>>>> a single flag bit, or large field with a counter or sequence > >>>>>>>>> number, or some kind of flow ID? If not a single flag, how > >>>>>>>>> large do you see the field > >>>>>>>> being? > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> If it is more than a flag (and I assume it would be), and is not > >>>>>>>>> mandatory for all implementations, then it seems to fall into > >>>>>>>>> the category of optional extensions. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Thanks, Larry > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> From: Haoweiguo <[email protected]> > >>>>>>>>> Date: Tuesday, November 11, 2014 10:18 AM > >>>>>>>>> To: Greg Mirsky <[email protected]> > >>>>>>>>> Cc: "[email protected]" <[email protected]> > >>>>>>>>> Subject: [nvo3] 答复: Comments on NVO3 data plane requirements > >>> for > >>>>>>>>> OAM > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Hi Greg, > >>>>>>>>> > >>>>>>>>> I fully agree with you. > >>>>>>>>> > >>>>>>>>> The real time OAM is passive performance measurement methods. I > >>>>>>>>> would like > >>>>>>>>> NVO3 data encapsulation has a field for marking and not affect > >>>>>>>>> forwarding of packets, the marking field is only used for > >>>>>>>>> performance measurement. The > >>>>>>>>> NVO3 packet with this marking flag don't need to be sent to > >>>>>>>>> control plane, it is different from OAM(ping/Trace) packet > >>>>>>>>>processing. > >>>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> > >>>>>>>>> weiguo > >>>>>>>>> > >>>>>>>>> ________________________________ > >>>>>>>>> > >>>>>>>>> 发件人: Greg Mirsky [[email protected]] > >>>>>>>>> 发送时间: 2014年11月12日 4:07 > >>>>>>>>> 收件人: Haoweiguo > >>>>>>>>> 抄送: [email protected] > >>>>>>>>> 主题: Re: [nvo3] Comments on NVO3 data plane requirements for > >>> OAM > >>>>>>>>> > >>>>>>>>> Hi Weiguo, > >>>>>>>>> > >>>>>>>>> marking groups of packets that belong to the particular flow to > >>>>>>>>> facilitate measurement of some performance metric, whether loss > >>>>>>>>> or delay/delay variation, may be viewed as one of passive > >>>>>>>>> performance > >>>>>>>> measurement methods. > >>>>>>>>> But such marking should not alter, at least not significantly > >>>>>>>>> alter, treatment of data flow in the network. Because of that, I > >>>>>>>>> believe, OAM flag should not be used for marking as that will > >>>>>>>>> force punting marked packets from fast forwarding path to the > >>>>>>>>> control plane. But it might be good to have a field in NVO3 > >>>>>>>>> header that may be used for marking and not affect forwarding of > >>> packets if altered. > >>>>>>>>> > >>>>>>>>> Regards, > >>>>>>>>> > >>>>>>>>> Greg > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Tue, Nov 11, 2014 at 12:34 AM, Haoweiguo > >>>>>>>>> <[email protected]> > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> Hi All, > >>>>>>>>> > >>>>>>>>> I maybe not clearly said in today’s NVO3 meeting, pls allow me > >>>>>>>>> to reiterate the OAM data plane requirements on the mail list. > >>>>>>>>> > >>>>>>>>> Currently NVO3 data plane encapsulation only includes one OAM > >>>>>>>>> flag, it is used for Ping/Trace similar applications. This kind > >>>>>>>>> of OAM application is initiated by operators for network > >>>>>>>>> connectivity verification, normally when network failure occurs. > >>>>>>>>> There is another OAM requirements of real time OAM or > >>>>>>>>> synthesizing OAM. It can be used for > >>>>>>>> packet loss detection in real time. > >>>>>>>>> When ingress NVE receives traffic from local TS, it gets packet > >>>>>>>>> statistics, and mark(coloring) the OAM flag relying on local > >>>>>>>>> policy when it performs > >>>>>>>>> NVO3 encapsulation. When egress NVEs receives the traffic, it > >>>>>>>>> decapsulates > >>>>>>>>> NVO3 encapsulation, and gets packet statistics with the real > >>>>>>>>> time OAM flag marking. By comparing the packet number of ingress > >>>>>>>>> NVE and the sum of all egress NVEs, packet loss can be deduced. > >>>>>>>>> This method can be applicable for both unicast and multicast > >>>>>>>>> traffic. Local policy on ingress NVE is configured by operators > >>>>>>>>> or automatically acquired from centralized orchestration. > >>>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> > >>>>>>>>> weiguo > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> _______________________________________________ > >>>>>>>>> nvo3 mailing list > >>>>>>>>> [email protected] > >>>>>>>>> https://www.ietf.org/mailman/listinfo/nvo3 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> _______________________________________________ > >>>>>>>>> nvo3 mailing list > >>>>>>>>> [email protected] > >>>>>>>>> https://www.ietf.org/mailman/listinfo/nvo3 > >>>>>>> _______________________________________________ > >>>>>>> nvo3 mailing list > >>>>>>> [email protected] > >>>>>>> https://www.ietf.org/mailman/listinfo/nvo3 > >>>> _______________________________________________ > >>>> nvo3 mailing list > >>>> [email protected] > >>>> https://www.ietf.org/mailman/listinfo/nvo3 > >>> > >>> _______________________________________________ > >>> nvo3 mailing list > >>> [email protected] > >>> https://www.ietf.org/mailman/listinfo/nvo3 > >> _______________________________________________ > >> nvo3 mailing list > >> [email protected] > >> https://www.ietf.org/mailman/listinfo/nvo3 > > > > > >
_______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
