Hello Deepak et al., so this sounds like we need more than just a (2nd) bit for delay measurement. Seems we need an optional header extension or a TLV to carry all the information (timestamps, oam Subtype). Sounds definitely more than a 32/64bit header could carry (*).
The optional header extension, when done similar to GUE, has a fixed position. For the TLV this would be an additional requirement. This would allow for hardware-stamping. Now if we introduce such an OAM extension header it could as well carry the "first" bit we discussed for packet loss measurement (?). Regards, Marc (*: at least all proposals so far have a base header that fits into 32/64 bit, plus IP and potential UDP) On Sun, 16 Nov 2014 16:44:54 +0000, Deepak Kumar (dekumar) wrote: > Hi, > > Please see inline +++DK: > > On 11/14/14 11:09 AM, "Jon Hudson" <[email protected]> wrote: > >> >> One comment in line.... >> >>> On Nov 13, 2014, at 11:47 PM, Vero Zheng <[email protected]> wrote: >>> >>> Hi Tom, >>> >>> Please see in-line. >>> >>> BR, Vero >>> >>>> -----Original Message----- >>>> From: nvo3 [mailto:[email protected]] On Behalf Of Tom Herbert >>>> Sent: Friday, November 14, 2014 4:27 PM >>>> To: Mach Chen >>>> Cc: Greg Mirsky; Haoweiguo; Marc Binderberger; Larry Kreeger; >>>> [email protected] >>>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane requirements for >>>> OAM >>>> >>>> On Wed, Nov 12, 2014 at 5:13 PM, Mach Chen <[email protected]> >>>> wrote: >>>>> Hi Tom, >>>>> >>>>>> -----Original Message----- >>>>>> From: Tom Herbert [mailto:[email protected]] >>>>>> Sent: Thursday, November 13, 2014 3:11 AM >>>>>> To: Marc Binderberger >>>>>> Cc: Mach Chen; Greg Mirsky; Haoweiguo; [email protected]; Larry Kreeger >>>>>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane requirements for >>>>>> OAM >>>>>> >>>>>> On Wed, Nov 12, 2014 at 2:11 AM, Marc Binderberger <[email protected]> >>>> wrote: >>>>>>> Hello Mach, >>>>>>> >>>>>>> so for delay measurement you use the color flag to mark a single >>>>>>> packet, which helps the receiver to pick the right packet? And >>>>>>> repeat this every time period T ? >>>>>>> >>>>>>> ...000100000010000001000... >>>>>> Is there there a draft or description of how this algorithm would >>>>>> work? Seems like there would need to be quite a bot of >>>>>> synchronization needed between end points (synchronized clocks, >>>>>> provisions to correlate measurements correctly with lost packets, >>>>>> replicated packets, etc.). Also, what is envisioned for range for >>>>>> the period? >>>>> >>>>> Here is a reference >>>> >>>> https://datatracker.ietf.org/doc/draft-chen-ippm-coloring-based-ipfpm-fr >>>> amew >>>> ork/. >>>> >>>> Thanks for the pointer. Regarding the need for synchronized clocks to >>>> measure >>>> delay, I consulted our local NTP expert. The host clock jitter we >>>> currently see in >>>> our network is currently usually greater than one-way packet delay (in >>>> some >>>> cases much greater), so in his words: >>>> "measuring one-way packet delays using host clocks is a lost cause". >>>> Please take this as just one data point! >> >> <Jon> Thank you. As someone who has managed NTP more times and for more >> years than I care to admit, this is a very good datapoint to consider. >> NTP helps many understand that time is relative. > > +++DK: As per our experience in carrier Ethernet we supported one way > delay and never found NTP useful even for our lab networks (I am referring > software based NTP NTPv3). > As mentioned below IEEE 1588v2 will vary based on equipment and operator > networks but in our testing we found it very precise if properly deployed. > IEEE 1588v2 is very precise if phy based timestamping is used. Even > timestamping at NP level provided great results for one way delay. > > If we want to accurately measure two way delay we need 4 timestamp total > on receiver of frame (this is to avoid processing time that's taken for > reply by software as hardware can put timestamp at lower layer without > doing delay and jitter calculation). > For one way delay we will require 2 timestamp, so lower layer hardware can > timestamp before packet is punted to software. > > As mentioned below I agree 8 byte IEEE 1588 timestamp is required. > > We should also look for Synthetic OAM applicability for performance ('O' > bit can be overloaded to do both Fault and performance if OAM is defined > with different oam Subtype for Delay and Loss frames and it will not be > too deep hardware inspection) as that give large flexibility > (synthetic/real loss measurement, Availability/unavailability, on-demand > and pro-active performance) and can be run on all flows of ECMP. > > Thanks, > Deepak >> >> >>> >>> [Vero] Thanks for this. What about the current experience with 1588v2 >>> then? >>>> >>>>> Yes, it does need some synchronization. As for the range, it depends >>>>> on two >>>> factors, one is the implementation limitation, the other the >>>> requirement of the >>>> operators. In the above reference, the suggested periods are 1s, 10s, >>>> 1min, >>>> 10min and 1h. >>>> I think if we were implementing delay measurement in GUE, I would >>>> advocate >>>> add a 64 bit optional field for timestamp, probably containing source >>>> time >>>> stamp, and echoed timestamp for a flow (usec resolution and similar in >>>> design >>>> TCP timestamp option). This easily gives a precise RTT, and if clocks >>>> are >>>> precisely synchronized then one way latency could be calculated also. >>> [Vero] If the source timestamp could be carried, it could also be used >>> for packet loss calculation/correlation. >>> >>>> Thanks, >>>> Tom >>>> >>>>> Best regards, >>>>> Mach >>>>>> >>>>>> Thanks, >>>>>> Tom >>>>>> >>>>>>> >>>>>>> One question I still have is: why is the measurement done in the NVE >>>> header? >>>>>>> The outer header is IP/IPv6, so couldn't we use the coloring for >>>>>>> the >>>>>>> IP/IPv6 header, assuming this is defined? >>>>>>> >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> Marc >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On Wed, 12 Nov 2014 09:34:52 +0000, Mach Chen wrote: >>>>>>>> Hi Tom, >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Tom Herbert [mailto:[email protected]] >>>>>>>>> Sent: Wednesday, November 12, 2014 5:06 PM >>>>>>>>> To: Mach Chen >>>>>>>>> Cc: Greg Mirsky; Haoweiguo; [email protected]; Larry Kreeger >>>>>>>>> (kreeger) >>>>>>>>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane requirements >>>>>>>>> for OAM >>>>>>>>> >>>>>>>>> On Wed, Nov 12, 2014 at 12:55 AM, Mach Chen >>>>>>>>> <[email protected]> >>>>>>>>> wrote: >>>>>>>>>> Hi Greg and all, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Single bit is not sufficient if someone wants to perform loss >>>>>>>>>> and delay measurement simultaneously, then two bits needed. >>>>>>>>> Is that necessary? Can they share the same time quantum (as well >>>>>>>>> as other metrics maybe to be added later)? In all the protocols >>>>>>>>> mentioned, the reserved bits are a somewhat precious resource. >>>>>>>> >>>>>>>> Yes, it's necessary if there is ECMP. >>>>>>>> >>>>>>>> Given one bit is used for both loss and delay measurement, for >>>>>>>> loss measurement, it periodically set and clear the marking bit, a >>>>>>>> flow is divided into consecutive blocks, and then the counting and >>>>>>>> calculating are based on each block. This is fine for loss >>>>>>>> measurement. >>>>>>>> >>>>>>>> For delay measurement, it has to make sure the timestamps >>>>>>>> (collected at sender and receiver) are for the same packet. >>>>>>>> Presumably, the time when changing the marking bit is right time >>>>>>>> to get >>>> the timestamps. >>>>>>>> Since there is ECMP, the first packet of a block at the sender may >>>>>>>> probably different from the first packet at the receiver, thus it >>>>>>>> will get the mismatched timestamps to calculate the delay. >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Mach >>>>>>>>> >>>>>>>>> Tom >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> >>>>>>>>>> Mach >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> From: nvo3 [mailto:[email protected]] On Behalf Of Greg >>>>>>>>>> Mirsky >>>>>>>>>> Sent: Wednesday, November 12, 2014 8:05 AM >>>>>>>>>> To: Haoweiguo >>>>>>>>>> Cc: [email protected]; Larry Kreeger (kreeger) >>>>>>>>>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane >>>> requirements >>>>>>>>>> for OAM >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Dear All, >>>>>>>>>> agree with Weiguo, single bit flag in fixed position would be >>>>>>>>>> sufficient and HW-friendly. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Greg >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Nov 11, 2014 at 3:51 PM, Haoweiguo >>>>>>>>>> <[email protected]> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi Larry, >>>>>>>>>> >>>>>>>>>> For marking purpose, i think one bit maybe OK, fixed fields in >>>>>>>>>> NVO3 header is precious. I would like it is set in fixed field, >>>>>>>>>> rather than in option field. Because chipset normally can't >>>>>>>>>> process optional field, it is hard to realize in-band >>>>>>>>>> performance measurement if using optional >>>>>>>>> field for marking. >>>>>>>>>> For other real time congestion control function, maybe more bits >>>>>>>>>> are needed. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> weiguo >>>>>>>>>> >>>>>>>>>> ________________________________ >>>>>>>>>> >>>>>>>>>> 发件人: Larry Kreeger (kreeger) [[email protected]] >>>>>>>>>> 发送时间: 2014年11月12日 4:33 >>>>>>>>>> 收件人: Haoweiguo; Greg Mirsky >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 抄送: [email protected] >>>>>>>>>> 主题: Re: [nvo3] Comments on NVO3 data plane requirements for >>>> OAM >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Weiguo, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> What do you envision this marking looking like? e.g. is it just >>>>>>>>>> a single flag bit, or large field with a counter or sequence >>>>>>>>>> number, or some kind of flow ID? If not a single flag, how >>>>>>>>>> large do you see the field >>>>>>>>> being? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> If it is more than a flag (and I assume it would be), and is not >>>>>>>>>> mandatory for all implementations, then it seems to fall into >>>>>>>>>> the category of optional extensions. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, Larry >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> From: Haoweiguo <[email protected]> >>>>>>>>>> Date: Tuesday, November 11, 2014 10:18 AM >>>>>>>>>> To: Greg Mirsky <[email protected]> >>>>>>>>>> Cc: "[email protected]" <[email protected]> >>>>>>>>>> Subject: [nvo3] 答复: Comments on NVO3 data plane requirements >>>> for >>>>>>>>>> OAM >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Greg, >>>>>>>>>> >>>>>>>>>> I fully agree with you. >>>>>>>>>> >>>>>>>>>> The real time OAM is passive performance measurement methods. I >>>>>>>>>> would like >>>>>>>>>> NVO3 data encapsulation has a field for marking and not affect >>>>>>>>>> forwarding of packets, the marking field is only used for >>>>>>>>>> performance measurement. The >>>>>>>>>> NVO3 packet with this marking flag don't need to be sent to >>>>>>>>>> control plane, it is different from OAM(ping/Trace) packet >>>>>>>>>> processing. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> weiguo >>>>>>>>>> >>>>>>>>>> ________________________________ >>>>>>>>>> >>>>>>>>>> 发件人: Greg Mirsky [[email protected]] >>>>>>>>>> 发送时间: 2014年11月12日 4:07 >>>>>>>>>> 收件人: Haoweiguo >>>>>>>>>> 抄送: [email protected] >>>>>>>>>> 主题: Re: [nvo3] Comments on NVO3 data plane requirements for >>>> OAM >>>>>>>>>> >>>>>>>>>> Hi Weiguo, >>>>>>>>>> >>>>>>>>>> marking groups of packets that belong to the particular flow to >>>>>>>>>> facilitate measurement of some performance metric, whether loss >>>>>>>>>> or delay/delay variation, may be viewed as one of passive >>>>>>>>>> performance >>>>>>>>> measurement methods. >>>>>>>>>> But such marking should not alter, at least not significantly >>>>>>>>>> alter, treatment of data flow in the network. Because of that, I >>>>>>>>>> believe, OAM flag should not be used for marking as that will >>>>>>>>>> force punting marked packets from fast forwarding path to the >>>>>>>>>> control plane. But it might be good to have a field in NVO3 >>>>>>>>>> header that may be used for marking and not affect forwarding of >>>> packets if altered. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Greg >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Nov 11, 2014 at 12:34 AM, Haoweiguo >>>>>>>>>> <[email protected]> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi All, >>>>>>>>>> >>>>>>>>>> I maybe not clearly said in today’s NVO3 meeting, pls allow me >>>>>>>>>> to reiterate the OAM data plane requirements on the mail list. >>>>>>>>>> >>>>>>>>>> Currently NVO3 data plane encapsulation only includes one OAM >>>>>>>>>> flag, it is used for Ping/Trace similar applications. This kind >>>>>>>>>> of OAM application is initiated by operators for network >>>>>>>>>> connectivity verification, normally when network failure occurs. >>>>>>>>>> There is another OAM requirements of real time OAM or >>>>>>>>>> synthesizing OAM. It can be used for >>>>>>>>> packet loss detection in real time. >>>>>>>>>> When ingress NVE receives traffic from local TS, it gets packet >>>>>>>>>> statistics, and mark(coloring) the OAM flag relying on local >>>>>>>>>> policy when it performs >>>>>>>>>> NVO3 encapsulation. When egress NVEs receives the traffic, it >>>>>>>>>> decapsulates >>>>>>>>>> NVO3 encapsulation, and gets packet statistics with the real >>>>>>>>>> time OAM flag marking. By comparing the packet number of ingress >>>>>>>>>> NVE and the sum of all egress NVEs, packet loss can be deduced. >>>>>>>>>> This method can be applicable for both unicast and multicast >>>>>>>>>> traffic. Local policy on ingress NVE is configured by operators >>>>>>>>>> or automatically acquired from centralized orchestration. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> weiguo >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> nvo3 mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://www.ietf.org/mailman/listinfo/nvo3 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> nvo3 mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://www.ietf.org/mailman/listinfo/nvo3 >>>>>>>> _______________________________________________ >>>>>>>> nvo3 mailing list >>>>>>>> [email protected] >>>>>>>> https://www.ietf.org/mailman/listinfo/nvo3 >>>>> _______________________________________________ >>>>> nvo3 mailing list >>>>> [email protected] >>>>> https://www.ietf.org/mailman/listinfo/nvo3 >>>> >>>> _______________________________________________ >>>> nvo3 mailing list >>>> [email protected] >>>> https://www.ietf.org/mailman/listinfo/nvo3 >>> _______________________________________________ >>> nvo3 mailing list >>> [email protected] >>> https://www.ietf.org/mailman/listinfo/nvo3 >> >> > > _______________________________________________ > nvo3 mailing list > [email protected] > https://www.ietf.org/mailman/listinfo/nvo3 _______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
