On Tue, Apr 17, 2018 at 12:51 AM, Frank Brockners (fbrockne) <fbroc...@cisco.com> wrote: > > Hi Tianran, > > Tom's note already includes the hint: You'll add IOAM data to the > protocol/layer that you're interested in monitoring. Again using Geneve over > IPv6 as an example: > * If you're interested in the overlay, i.e. Geneve (e.g. timestamping the > packet when it enters and exists the tunnel) - you'd add IOAM data to Geneve > * If you're interested in the underlay, i.e. IPv6 (e.g. you'd like to > understand which path packets take in the v6 network) - you'd add IOAM data > to IPv6 > * If you're interested in both, then you'd add IOAM data to Geneve and IPv6 > Frank,
In that case why not just use a hop-by-hop option for measuring the underlay and a destination option for measuring the overlay? The advantage is that this works _any_ IP encapsulation method or any IP protocol for that matter. I don't believe adding ippm to every encapsulation protocol is straightforward: e.g. draft-brockners-ippm-ioam-geneve describe but notes the limited size of header, draft-weis-ippm-ioam-gre states that a new EtherType would be needed just for this purpose. This also entails additional encapsulation-specific HW support also, whereas support destination and hbh options could be more generic. Tom > Draft draft-ietf-ippm-ioam-data-02 already mentions layering (see section 3): > "Layering: If several encapsulation protocols (e.g., in case of tunneling) > are stacked on top of each other, IOAM data-records could be present at every > layer. The behavior follows the ships-in-the-night model." > > Given the discussion here, we'll add some additional text in the next > revision to make things crisper (e.g. adding an example might help). > > Frank > > -----Original Message----- > From: Tianran Zhou <zhoutian...@huawei.com> > Sent: Dienstag, 17. April 2018 03:18 > To: Tom Herbert <t...@herbertland.com> > Cc: Shwetha Bhandari (shwethab) <shwet...@cisco.com>; Frank Brockners > (fbrockne) <fbroc...@cisco.com>; Mickey Spiegel > <mspie...@barefootnetworks.com>; NVO3 <nvo3@ietf.org>; Service Function > Chaining IETF list <s...@ietf.org>; IETF IPPM WG <i...@ietf.org> > Subject: RE: [ippm] [Int-area] encapsulation of IOAM data in various > protocols - follow up from WG discussion in London > > I think it's better that Frank or Shwetha can explain the multi-layer use > case in detail. > > Tianran >> -----Original Message----- >> From: Tom Herbert [mailto:t...@herbertland.com] >> Sent: Monday, April 16, 2018 10:40 PM >> To: Tianran Zhou <zhoutian...@huawei.com> >> Cc: Shwetha Bhandari (shwethab) <shwet...@cisco.com>; Frank Brockners >> (fbrockne) <fbroc...@cisco.com>; Mickey Spiegel >> <mspie...@barefootnetworks.com>; NVO3 <nvo3@ietf.org>; int-area >> <int-a...@ietf.org>; Service Function Chaining IETF list >> <s...@ietf.org>; IETF IPPM WG <i...@ietf.org> >> Subject: Re: [ippm] [Int-area] encapsulation of IOAM data in various >> protocols - follow up from WG discussion in London >> >> On Mon, Apr 16, 2018 at 6:31 AM, Tianran Zhou <zhoutian...@huawei.com> wrote: >> > Hi Shwetha, >> > >> > You are talking about the outer encapsution. It is straight forward >> > for the underlay to record by the header. But what about the >> > overlay, i.e., inner encapsulation(e.g. geneve)? Without special >> > configuration, intermediate node will not read the inner header, >> > hence not be able to process IOAM.e >> >> Hi Tianran, >> >> I believe that is also not protocol conformant. Intermediate nodes >> should not be processing transport layer data as this can lead to >> misinterpretation and possibly silent data corruption. >> >> For instance, Geneve is a UDP encapsulation protocol with assigned port 6081. >> In order for an intermediate device to process the Geneve >> encapsulation header it would need to look for packets with >> destination port of 6081 since that is the only possible >> discriminator. However, transport port numbers do not have global >> meaning and hosts may use port numbers for other purposes (RFC7605 >> describes this). So a packet to port 6081 might be something other >> than Geneve and may be misinterpreted. If a misinterpreted packet is changed >> (like ippm data is written) then that would be systematic silent data >> corruption. >> >> As far as I know, hop-by-hop options is the only protocol confirming >> mechanism that allows an intermediate note to change data of packet in >> flight. >> Encpasulation is the only conforming mechanism that allows an >> intermediate node to add data (like extension headers) to a packet in flight. >> >> Tom >> >> > Maybe we are not synced by this overlay/underlay use case. :-) >> > >> > Tianran >> > >> > >> > >> > ________________________________ >> > Sent from WeLink >> > >> > 发件人： Shwetha Bhandari (shwethab) >> > 收件人： Tianran Zhou<zhoutian...@huawei.com>;Frank Brockners >> > (fbrockne)<fbroc...@cisco.com>;Mickey >> > Spiegel<mspie...@barefootnetworks.com>;Tom >> > Herbert<t...@herbertland.com> >> > 抄送： NVO3<nvo3@ietf.org>;int-area<int-a...@ietf.org>;Service Function >> > Chaining IETF list<s...@ietf.org>;IETF IPPM WG<i...@ietf.org> >> > 主题： Re: [ippm] [Int-area] encapsulation of IOAM data in various >> > protocols - follow up from WG discussion in London >> > 时间： 2018-04-16 18:17:01 >> > >> > Hi Tianran, >> > >> >> If I recall right, it is not written in the ioam data draft. >> > >> > Data draft is defining the data to be carried in IOAM in an >> > encapsulation agnostic way, it does not specify how the >> > encapsulation protocol is configured. >> > >> > >> > >> >> Yes, node by node configuration is an easy way. >> > >> > While it is, it does not have to be a node by node configuration. It >> > can be part of the encapsulation definition. >> > >> > For e.g. If the encapsulation is IPv6 and if we define the data to >> > be carried as HbH options, then based on the Option Type with >> > highest order 2 bits set to 00 then the v6 nodes that implement IOAM >> > will process the option and others will skip over. >> > >> > >> > >> > >> > >> > Thanks, >> > >> > Shwetha >> > >> > >> > >> > From: ippm <ippm-boun...@ietf.org> on behalf of Tianran Zhou >> > <zhoutian...@huawei.com> >> > Date: Monday, April 16, 2018 at 2:36 PM >> > To: "Frank Brockners (fbrockne)" <fbroc...@cisco.com>, Mickey >> > Spiegel <mspie...@barefootnetworks.com>, Tom Herbert >> > <t...@herbertland.com> >> > Cc: NVO3 <nvo3@ietf.org>, "int-a...@ietf.org" <int-a...@ietf.org>, >> > Service Function Chaining IETF list <s...@ietf.org>, IETF IPPM WG >> > <i...@ietf.org> >> > Subject: Re: [ippm] [Int-area] encapsulation of IOAM data in various >> > protocols - follow up from WG discussion in London >> > >> > >> > >> > Hi Frank, >> > >> > >> > >> > If I recall right, it is not written in the ioam data draft. >> > >> > Yes, node by node configuration is an easy way. In the >> > draft-zhou-ippm-ioam-yang, we have the “protocol-type” to indicate >> > the layering. >> > >> > +--rw ioam >> > >> > +--rw ioam-profiles >> > >> > +--rw enabled? boolean >> > >> > +--rw ioam-profile* [profile-name] >> > >> > +--rw profile-name string >> > >> > +--rw filter >> > >> > | +--rw filter-type? ioam-filter-type >> > >> > | +--rw acl-name? -> /acl:acls/acl/name >> > >> > +--rw protocol-type? ioam-protocol-type >> > >> > +--rw incremental-tracing-profile {incremental-trace}? >> > >> > | ... >> > >> > +--rw preallocated-tracing-profile {preallocated-trace}? >> > >> > | ... >> > >> > +--rw pot-profile {proof-of-transit}? >> > >> > | ... >> > >> > +--rw e2e-profile {edge-to-edge}? >> > >> > ... >> > >> > >> > >> > >> > >> > Tianran >> > >> > From: Frank Brockners (fbrockne) [mailto:fbroc...@cisco.com] >> > Sent: Monday, April 16, 2018 4:51 PM >> > To: Tianran Zhou <zhoutian...@huawei.com>; Mickey Spiegel >> > <mspie...@barefootnetworks.com>; Tom Herbert <t...@herbertland.com> >> > Cc: NVO3 <nvo3@ietf.org>; int-a...@ietf.org; Service Function >> > Chaining IETF list <s...@ietf.org>; IETF IPPM WG <i...@ietf.org> >> > Subject: RE: [ippm] [Int-area] encapsulation of IOAM data in various >> > protocols - follow up from WG discussion in London >> > >> > >> > >> > Hi Tianran, >> > >> > >> > >> > IOAM is a domain specific feature (see also >> > draft-ietf-ippm-ioam-data-02 sections 3 and 4), which allows an >> > operator to control by means of configuration where and for which >> > traffic IOAM data fields are added/updated/removed from the customer >> > traffic. Using your example of Geneve over IPv6 – with IOAM data in >> > both the Geneve and the IPv6 protocol, one would expect that the >> > operator configures the endpoints of the Geneve tunnel to operate on >> > the IOAM data in Geneve, and the IPv6 routers that the Geneve tunnel >> traverses to operate on the IOAM data in IPv6. >> > >> > >> > >> > Frank >> > >> > >> > >> > From: Tianran Zhou <zhoutian...@huawei.com> >> > Sent: Montag, 16. April 2018 10:37 >> > To: Frank Brockners (fbrockne) <fbroc...@cisco.com>; Mickey Spiegel >> > <mspie...@barefootnetworks.com>; Tom Herbert <t...@herbertland.com> >> > Cc: NVO3 <nvo3@ietf.org>; int-a...@ietf.org; Service Function >> > Chaining IETF list <s...@ietf.org>; IETF IPPM WG <i...@ietf.org> >> > Subject: RE: [ippm] [Int-area] encapsulation of IOAM data in various >> > protocols - follow up from WG discussion in London >> > >> > >> > >> > Hi Frank, >> > >> > >> > >> > How does a forwarder know when and where to insert the data? >> > >> > In the case of Geneve over IPv6, do you mean the device need to scan >> > all the protocol stack? Or just the outer encapsulation? >> > >> > >> > >> > Tianran >> > >> > >> > >> > From: ippm [mailto:ippm-boun...@ietf.org] On Behalf Of Frank >> > Brockners >> > (fbrockne) >> > Sent: Monday, April 16, 2018 3:08 PM >> > To: Mickey Spiegel <mspie...@barefootnetworks.com>; Tom Herbert >> > <t...@herbertland.com> >> > Cc: NVO3 <nvo3@ietf.org>; int-a...@ietf.org; Service Function >> > Chaining IETF list <s...@ietf.org>; IETF IPPM WG <i...@ietf.org> >> > Subject: Re: [ippm] [Int-area] encapsulation of IOAM data in various >> > protocols - follow up from WG discussion in London >> > >> > >> > >> > >> > >> > Tom, >> > >> > >> > >> > a quick addition to what Mickey mentioned below: What you seem to >> > have in mind is what draft-ietf-ippm-ioam-data-02 refers to as “layering” >> > (see section 3.), i.e. if you’re running for example Geneve over >> > IPv6, then IOAM data could be encapsulated in both protocols, Geneve >> > and >> > IPv6 – giving you visibility into the “underlay” (IPv6) and the “overlay” >> (Geneve). >> > >> > >> > >> > Frank >> > >> > >> > >> > From: ippm <ippm-boun...@ietf.org> On Behalf Of Mickey Spiegel >> > Sent: Freitag, 13. April 2018 20:22 >> > To: Tom Herbert <t...@herbertland.com> >> > Cc: NVO3 <nvo3@ietf.org>; int-a...@ietf.org; Service Function >> > Chaining IETF list <s...@ietf.org>; IETF IPPM WG <i...@ietf.org> >> > Subject: Re: [ippm] [Int-area] encapsulation of IOAM data in various >> > protocols - follow up from WG discussion in London >> > >> > >> > >> > Tom, >> > >> > >> > >> > On Thu, Apr 12, 2018 at 10:17 PM, Tom Herbert <t...@herbertland.com> wrote: >> > >> > Mickey, >> > >> > Looking at these ippm drafts more closely, I have a much more >> > fundamental concern. >> > >> > In draft-brockners-ippm-ioam-geneve-00 for instance, there is the >> > text in the introduction: >> > >> > "In-situ OAM (IOAM) records OAM information within the packet while >> > the packet traverses a particular network domain. The term "in-situ" >> > refers to the fact that the IOAM data fields are added to the data >> > packets rather than is being sent within packets specifically >> > dedicated to OAM. This document defines how IOAM data fields are >> > transported as part of the Geneve [I-D.ietf-nvo3-geneve] >> > encapsulation." >> > >> > I assume this means that as packets with Geneve encapsulation >> > traverse the network they are interpreted by intermediate nodes as >> > being Geneve. Since Geneve is a UDP encapsulation, then the >> > destination UDP port number would be used to identify packets as >> > being Geneve. So an intermediate device might be looking for UDP >> > packets destined to port >> > 6081 (the assigned UDP port for Geneve). If my understanding is >> > correct, then this is a problem. >> > >> > UDP port numbers do not have global meaning. An intermediate device >> > may very well see UDP packets destined to port 6081 that are not >> > actually Geneve. This scenario is discussed in RFC7605: >> > >> > "...intermediate device interprets traffic based on the port number. >> > It is important to recognize that any interpretation of port numbers >> > -- except at the endpoints -- may be incorrect, because port numbers >> > are meaningful only at the endpoints." >> > >> > If the UDP data is modified, as the draft would imply, then >> > misinterpretation may also mean silent data corruption of packets. A >> > protocol that would allow this seems pretty incorrect! Note that >> > this would be true also for any UDP encapsulation that the network >> > tries to interpret. >> > >> > >> > >> > The intention is to allow for multiple nodes that a packet traverses >> > >> > to be able to insert IOAM node information in the same trace option, >> > >> > but leave some flexibility regarding which nodes actually do the >> > >> > IOAM processing and the node information. This may vary >> > >> > depending on the transport. >> > >> > >> > >> > In case of a tunneled encapsulation such as Geneve or VXLAN, >> > >> > there may still be multiple hops. For example a network may use >> > >> > Geneve or VXLAN, but only do L2 processing at ToRs, with L3 >> > >> > processing done at aggregation or core switches. In this case >> > >> > many packets would do 2 Geneve or VXLAN hops, so the packet >> > >> > would contain IOAM node information from two nodes. >> > >> > >> > >> > Another example is service function chaining using Geneve or >> > >> > VXLAN rather than NSH. >> > >> > >> > >> > >> > I am also wondering if hop-by-hop options been considered for this >> > application? Their interpretation in the network is unabiguous and >> > they also have the advantage that the work with any IP protocol or >> > encapsulation. >> > >> > >> > >> > IPv6 hop-by-hop options has been considered. See >> > >> > draft-brockners-inband-oam-transport-05. This has not yet been >> > >> > broken out into a separate draft. >> > >> > >> > >> > Mickey >> > >> > >> > >> > >> > Thanks, >> > Tom >> > >> > >> > On Thu, Apr 12, 2018 at 3:31 PM, Mickey Spiegel >> > <mspie...@barefootnetworks.com> wrote: >> > >> >> Tom, >> >> >> >> On Thu, Apr 12, 2018 at 2:46 PM, Tom Herbert <t...@herbertland.com> wrote: >> >>> >> >>> On Thu, Apr 12, 2018 at 9:54 AM, Greg Mirsky >> >>> <gregimir...@gmail.com> >> >>> wrote: >> >>> > Hi Frank, >> >>> > thank you for sharing your points. Please find my notes in-line >> >>> > and tagged >> >>> > GIM>>. I believe that this is very much relevant to work of >> >>> > GIM>>other >> >>> > working >> >>> > groups that directly work on the overlay encapsulations in the >> >>> > center of the discussion and hence I've added them to the list. >> >>> > Hope we'll have more opinions to reach the conclusion that is >> >>> > acceptable to all. >> >>> > >> >>> > Regards, >> >>> > Greg >> >>> > >> >>> > On Wed, Apr 11, 2018 at 12:02 PM, Frank Brockners (fbrockne) >> >>> > <fbroc...@cisco.com> wrote: >> >>> >> >> >>> >> Back at the IPPM meeting in London, we discussed several drafts >> >>> >> dealing with the encapsulation of IOAM data in various >> >>> >> protocols (draft-brockners-ippm-ioam-vxlan-gpe-00, >> >>> >> draft-brockners-ippm-ioam-geneve-00, >> >>> >> draft-weis-ippm-ioam-gre-00). One discussion topic that we >> >>> >> decided to take to the list was the question on whether >> >>> >> draft-ooamdt-rtgwg-ooam-header could be leveraged.. After >> >>> >> carefully considering draft-ooamdt-rtgwg-ooam-header, I came to >> >>> >> the conclusion that the “OOAM header” does not meet the needs >> >>> >> of >> >>> >> IOAM: >> >>> >> >> >>> >> * Efficiency: IOAM adds data to live user traffic. As such, an >> >>> >> encapsulation needs to be as efficient as possible. The “OOAM header” >> >>> >> is 8 >> >>> >> bytes long. The approach for IOAM data encapsulation in the >> >>> >> above mentioned drafts only requires 4 bytes. Using the OOAM >> >>> >> header approach would add an unnecessary overhead of 4 bytes – >> >>> >> which is significant. >> >>> Greg, >> >>> >> >>> I'm missing something here. I looked at the drafts you referenced >> >>> and each of them looks like the overhead for OAM is greater that >> >>> four bytes. In each there is some overhead equivalent to >> >>> type/length, for instance in Geneve four bytes are needed for >> >>> option class, type, and length. Unless the the OAM data is zero >> >>> length, I don't see how this adds up to only four bytes of overhead. >> >> >> >> >> >> The four versus eight bytes just refers to the fields in the four >> >> bytes of IOAM info, that is common to all IOAM options. Beyond >> >> that, there are IOAM option specific fields. For example if doing >> >> one of the IOAM trace options, there are four bytes of trace option >> >> header, including the IOAM-trace-type, NodeLen, Flags, and >> >> RemainingLen fields. These are followed by the node data list >> >> containing the per hop IOAM information. >> >> >> >> In looking at the OOAM header content, it has nothing to do with >> >> any of the IOAM information after the first four bytes. It contains >> >> another variant of the information in the first four bytes of IOAM >> >> info, spread out over eight bytes. >> >> >> >>> >> >>> Tom >> >>> >> >>> > >> >>> > GIM>> The difference in four octets is because OOAM Header: >> >>> > >> >>> > provides more flexibility, e.g. Flags field and Reserved fields; >> >> >> >> >> >> The flags field only has one defined flag at the moment, for a >> >> timestamp block. For IOAM trace we need per hop timestamps, which >> >> the timestamp block cannot address, i.e. the timestamp block is >> >> redundant for >> IOAM. >> >> >> >>> >> >>> > supports larger OAM packets than iOAM header; >> >> >> >> >> >> For IOAM purposes, 1020 octets is more than enough. >> >> >> >>> >> >>> > is future proof by supporting versioning (Version field). >> >> >> >> >> >> IMO, taking the first two bits of the IOAM-Type to define a Version >> >> field would be a good thing. This does not require adding four more >> >> bytes of overhead. 64 IOAM-Types is more than enough. >> >> >> >>> >> >>> >> >> >>> >> * Maturity: IOAM has several implementations, which were also >> >>> >> shown at recent IETF hackathons – and we’re expecting >> >>> >> additional implementations to be publicized soon. Interoperable >> >>> >> implementations need timely specifications. Despite the >> >>> >> question being asked, the recent thread on OOAM in the NVO3 >> >>> >> list hasn’t revealed any implementation of the OOAM header. >> >>> >> In >> >>> >> addition, the thread revealed that several fundamental >> >>> >> questions about the OOAM header are still open, such as whether >> >>> >> or how active OAM mechanisms within protocols such as Geneve >> >>> >> would apply to the OOAM header. This ultimately means that we >> >>> >> won’t get to a timely specification. >> >>> > >> >>> > GIM>> May I ask which encapsulations supported by the >> >>> > GIM>> implementations >> >>> > you >> >>> > refer to. Until very recently all iOAM proposals were to use >> >>> > meta-data TLV in, e.g. Geneve and NSH. And if these or some of >> >>> > these implementations already updated to the newly proposed iOAM >> >>> > shim, I don't see problem in making them use OOAM Header. Would >> >>> > you agree? >> >>> > >> >>> >> >> >>> >> * Scope: It isn’t entirely clear to which protocols the OOAM >> >>> >> header would ultimately apply to. The way the OOAM header is >> >>> >> defined, OOAM uses a 8-bit field for “Next Prot”, the next >> >>> >> protocol. Some protocols that IOAM data needs to be >> >>> >> encapsulated into use 16-bits for their next protocol code points. >> >>> >> See e.g. >> >>> >> the GRE encapsulation – as specified in >> >>> >> draft-weis-ippm-ioam-gre-00. >> >>> > >> >>> > GIM>> The first paragraph of the Introduction section states: >> >>> > New protocols that support overlay networks like VxLAN-GPE >> >>> > [I-D.ietf-nvo3-vxlan-gpe], GUE [I-D.ietf-nvo3-gue], Geneve >> >>> > [I-D.ietf-nvo3-geneve], BIER >> >>> > [I-D.ietf-bier-mpls-encapsulation], >> and >> >>> > NSH [I-D.ietf-sfc-nsh] support multi-protocol payload, e.g. >> >>> > Ethernet, IPv4/IPv6, and recognize Operations, Administration, and >> >>> > Maintenance (OAM) as one of distinct types. That ensures that >> >>> > Overlay OAM (OOAM)packets are sharing fate with Overlay data packet >> >>> > traversing the underlay. >> >>> > I'm updating the OOAM Header draft and along with cleaning nits >> >>> > will update reference to GUE. I think that the list and the >> >>> > statemnt are quite clear in identifying the scope of networks >> >>> > that may benefit from using not only common OOAM Header but >> >>> > common OOAM mechanisms, e.g. Echo Request/Reply. >> >>> > >> >>> >> With the above in mind, I’d suggest that the WG moves forward >> >>> >> with specific definitions for encapsulating IOAM data into >> >>> >> protocols – per the above mentioned drafts. >> >>> >> >> >>> >> >> >>> >> >> >>> >> Regards, Frank >> >>> >> >> >>> >> >> >>> >> _______________________________________________ >> >>> >> ippm mailing list >> >>> >> i...@ietf.org >> >>> >> https://www.ietf.org/mailman/listinfo/ippm >> >>> >> >> >>> > >> >>> > >> >>> > _______________________________________________ >> >>> > Int-area mailing list >> >>> > int-a...@ietf.org >> >>> > https://www.ietf.org/mailman/listinfo/int-area >> >>> > >> >>> >> >>> _______________________________________________ >> >>> ippm mailing list >> >>> i...@ietf.org >> >>> https://www.ietf.org/mailman/listinfo/ippm >> >> >> >> >> > >> > _______________________________________________ nvo3 mailing list nvo3@ietf.org https://www.ietf.org/mailman/listinfo/nvo3
