On 6/9/15 5:25 AM, Carlos Pignataro (cpignata) wrote:
Hi, RtgWG,
Please find below some review comments on draft-rtg-dt-encap-02.
https://datatracker.ietf.org/doc/draft-rtg-dt-encap/
While this is not a comprehensive fine-tooth-comb review, I hope these
(somewhat high-level) comments are useful.
Carlos,
Thanks for the comments and sorry for the delay in responding.
General comments:
Scanning through the document, I have two high-level concerns.
• First, there is really no evident or apparent definition of what
“Encapsulation” is. This is not a pedantic comment, but I think it is a root
cause of the point that follows.
• Many of the considerations on this document are not applicable to the
scoped “encapsulation” per se. Instead, they relate to a specific layer
underneath the encapsulation, to a layer that further encapsulates the
encapsulation.
For example, the document sets of explaining the need for understanding common
requirements and considerations among groups of encaps. In addition to encaps
in NOV3, these are also the BIER Header and NSH. Both these can be carried by
multiple underlays each with different characteristics. However, the document
conflates requirements and characteristics from the encap itself, and
intertwines them with characteristics and requirements on the underlay.
You must have read a different document, because draft-rtg-dt-encap
contains no requirements ;-)
More seriously, trying to come up with a strict definition of
"encapsulation" or placing general requirements across a broad set of
approaches would be quite futile since that tends to result in proxy
discussions which are really about "I don't want that requirement on my
protocol".
That is why the title and the content of the document "considerations".
It seems like folks appreciate having captured this broad set of
considerations in one place, and it might be useful *input* to working
groups that want to draft their own requirements and protocols.
I think this document would greatly benefit from a cleaner separation of what
happens at the encap, versus what is expected, required, or considered from the
underlay (transport, network, tunnel, etc). This is particularly exacerbated in
the ECMP and MTU sections, but present in others as well.
Take for example ECMP — the BIER header has an Entropy field. Different encaps
can provide Entropy fields to be used as a source for entropy, either at the
encap (e.g., service) layer or by an underlay. One example is L2TPv3 session id
and GRE Key (as per RFC 5640). However, the ECMP sections focus on IP and
IP/UDP methods, which frankly are applicable generally and not only to these
“Encapsulations”.
Note that the BIER entropy has nothing to do with the "transport"
entropy; we tried to capture that in the document to avoid confusion.
The BIER entropy is how BIER selects paths. In theory there can be an
MPLS entropy label in addition for the "transport" entropy.
We've tried to specify how the "transport" entropy fits in, and how that
relates to using IP/UDP and MPLS transports.
That seems to result useful information in the draft, because when we
started off the Design Team it wasn't clear to (at least us) how those
things relate; whether one should have a single entropy field even with
nested encaps etc.
If there are specific text in the considerations that you find unclear
or incorrect, then please point them out so we can collectively improve
the document.
And FWIW that is an ask from Alia and others that the WG add more
information about MPLS to the document.
What are your specific concerns about the MTU considerations?
AFAICT any device along the path which adds (or grows) a header in a
packet needs to be concerned about the MTU implications.
And I believe some of these potential disconnects go back to the high-level
concern #1, lack of definition/scope of what is an encap (and what is not).
Just to be clear, I am all for inter-layer cooperation and requirement
realization. However, if the goal is to understand encapsulation common
requirements, then a more clean separation of the encapsulation headers versus
a specific transport of said encapsulations will make it easier to understand
if those requirements belong in an encap or in an underlay.
FWIW my take is that if the folks working on different encapsulations in
different WGs consider what it in this document, then we can avoid
endless discussions of where to draw lines in the sand by working on
strict definitions.
A smaller one: this document is titled "Encapsulation Considerations” (without
any qualifiers), but the abstract narrowscopes it to a specific encaps.
Which encaps? The document focuses on three that are currently under
development in the IETF. (Perhaps your definitions of "encaps" is what
the document calls "transport", which would make our discussion rather
confusing?)
Some more specific comments, prefaced with “CMP":
2. Overview
…
o SFC carries service meta-data which might be modified or
unmodified as the packets follow the service path. SFC talks of
some loop avoidance mechanism which is likely to result in
modifications for for each hop in the service chain even if the
meta-data is unmodified.
CMP: This is not complete (incorrect by omission). The SFC Encap is responsible
first for Serfice Function Path Identification, and second, for Metadata. See
Section 1.3 and 4.1 of
https://tools.ietf.org/html/draft-ietf-sfc-architecture-09.
I'll reword it to include the Serfice Function Path Identification part.
o SFC is all about carrying service meta-data along a path, and
different services might need different types and amount of meta-
data.
CMP: It is true that metadata requires extensibility. However the SFC Encap is
all about path identification *and* carrying metadata.
CMP: Also, SFC’s extensibility has to do with OAM and graphs as well.
Ditto.
CMP: By the way, encaps need to strike a balance between flexibility and
performance. This point could be made more explicitly.
Suggestions how and where to do that and not make it sound like
motherhood and apple pie?
What would you say to make a reader learn something they might not have
already known about that tradeoff?
Most of the issues discussed in this document are not new. The IETF
and industry as specified and deployed many different encapsulation
or tunneling protocols over time, ranging from simple IP-in-IP and
GRE encapsulation, IPsec, pseudo-wires, session-based approached like
L2TP, and the use of MPLS control and data planes. IEEE 802 has also
defined layered encapsulation for Provider Backbone Bridges (PBB) and
IEEE 802.1Qbp (ECMP). This document tries to leverage what we
collectively have learned from that experience and summarize what
would be relevant for new encapsulations like NVO3, SFC, and BIER.
CMP: I think it would be much clearer here to specify terms around what is
encap, transport, etc. L2TPv3 for example can run over IPv6 directly, over
IP/UDP, etc. Those cases have key differences although the actual L2TPv3 header
does not change.
One criteria that I found useful in thinking about that to include and
exclude from the document was whether there is something non-obvious, or
something known in one part of the IETF but not known in other parts,
where sharing that knowledge would result in the IETF producing better
protocols. If you think strict definitions of encap vs. transport would
be an aid in such sharing and learning, then we should definitely get
together (e.g., in Prague) so I can understand.
3. Common Issues
…
o How to provide entropy for Equal Cost MultiPath (ECMP) routing
CMP: And LAGs?
LAG isn't something we tend to standardize in the IETF.
One could add a note saying "Note that the same entropy might also be
used at layer 2 e.g. for Link Aggregration (LAG)". But that belongs on
the text and not in the
o OAM - what support is needed in an encapsulation format?
CMP: OAM in what context? fault management, performance management, all?
The question is broad, hence "all".
CMP: Also, this section seems to lack an issue of “adaptation” of the payload.
Please explain further.
4. Scope
o Focus on the class of encapsulations that would run over IP/UDP.
That was done to avoid being distracted by the data-plane and
control-plane interaction, which is more significant for protocols
that are designed to run over "transports" that maintain session
or path state.
CMP: I am not sure if the relationship between running over “transports” (which
transports?) and control-plane - data-plane complexity is clear — it is not
clear to me at least.
This was the scope that the design team set up to improve its
probability of delivering a useful document on time.
Hence the "scope" section presumably needs to be done as the WG works on
this document.
7. Entropy
In many cases the encapsulation format needs to enable ECMP in
unmodified routers. Those routers might use different fields in TCP/
UDP packets to do ECMP without a risk of reordering a flow.
CMP: Is this an encap-supported requirements, or a requirement of IP/UDP in
general, whatever is transported?
I'm not sure I understand your question. When Dino defined the LISP
header he wanted LISP packets to take advantage of ECMP in unmodified
routers. That same desire exists for other encapsulations that have been
and are being defined in the IETF and elsewhere.
Note that the document isn't about requirements!
the ephemeral port range) plus the outer IP addresses which seems to
be sufficient for entropy; using outer IPv6 headers would give the
option for more entropy should it be needed in the future.
CMP: And the IPv6 Flow Label and RFC 6438?
The section discusses IPv6 flow label a few paragraphs later.
The design team didn't discuss RFC6438 explicitly, but it seems like the
approach (from LISP and VXLAN) to use a random UDP source port plus
outer IPv6 flow label is consistent with RFC 6438. But I'll at least add
a reference to RFC6438.
There is some interaction between entropy and OAM and extensibility
mechanism. It is desirable to be able to send OAM packets to follow
the same path as network packets. Hence OAM packets should use the
CMP: This is having the underlying assumption that “OAM” is “OAM packets” as
opposed to also fields in data packets. If OAM is in the packet, it fate-shares
by definition.
But it is more subtle than that. The difference of OAM frames and the
frames they should "monitor" must not affect the ECMP calculation. The
document (tries to) point this out. Please suggest improved text if that
is unclear.
Architecturally the entropy and the next header field are really part
of enclosing delivery header. UDP with entropy goes hand-in-hand
with the outer IP header. Thus the UDP entropy is present for the
CMP: This is a very important point, and this clarity in demarcation should be
throughout, to understand at which layer different requirements are.
8. Next-protocol indication
Next-protocol indications appear in three different context for
encapsulations.
…
Secondly, the encapsulation needs to indicate the type of its
payload, which is in scope for the design of the encapsulation.
CMP: I believe that this section should not start assuming an explicit protocol
indication. A demux context can provide protocol identification, or it could be
explicitly carrier (self-defining package). And perhaps that is the first
consideration.
Where does it assume it is explicit? Second paragraph has existing
examples of both explicit and implicit.
We
have existing protocols which use Ethernet types (such as GRE). Here
each encapsulation header can potentially makes its own choices
between:
o Reuse Ethernet types - makes it easy to carry existing L2 and L3
protocols including IPv6, IPv6, and Ethernet.
CMP: I do not believe that the options are Ethertype, IP protocol number, or
own registry only.
What are the other options? Are you suggesting an implicit
(session-based) identification?
That would assume some form of session identification.
The document mentions that possibility and ends with "*Encapsulations
might want to consider the tradeoffs between such more flexible versus
more fixed approaches."*
CMP: Nit — repeat IPv6.
Thanks.
In summary:
o Would it be useful for the IETF come up with a common scheme for
encapsulation protocols? If not each encapsulation can define its
own scheme.
CMP: This seems to be a non-sequitur. Why would that be useful?
First of all, it is a question and not a statement.
I think it would make sense for the broader IETF to consider the
cost/benefit tradeoffs.
There might be some benefits as we define X over Y, Y over X, X over W
over Y, etc (which we seemed to have done with most existing protocols
over the last few decades). But there would be some cost to defining
this namespace across different active WGs.
9. MTU and Fragmentation
CMP: This is another key section that could use clarify between encap and
transport.
See above.
In summary:
o In some deployments an encapsulation can assume well-managed MTU
hence no need for fragmentation and reassembly related to the
encapsulation.
o Even so, it makes sense for ingress to track any ICMP packet too
big addressed to ingress to be able to log any MTU
misconfigurations.
CMP: This section seems to be conflating MTU with PMTUD. Cleary related, but
those are different things. The area of pre-fragmentation vs. post-frag is also
underspecified. See e.g., https://tools.ietf.org/html/rfc3931#section-4.1.4
I don't think it is useful to talk about PMTU because the first bullet
is really about a common interface MTU across the whole underlay
network. That underlay network will have lots of different paths. Hence
to make it the text more specific I think one would need to introduce a
notion of a some "topology MTU" accross the underlay topology.
Introducing such a brand new concept and term doesn't seem to be
worth-while in explaining the issue and choices.
This document has a number of considerations; I certainly don't expect
it to specify all the details of how to handle MTU and frag/reass.
10. OAM
In terms of what we have heard from the various working groups there
seem to be needs to:
o Be able to send out-of-band OAM messages - that potentially should
follow the same path through the network as some flow of data
packets.
CMP: It is not clear what are “out-of-band messages” that follow the same path
as data packets…
TRILL has an example of this. Might exist elsewhere as well.
CMP: Also, this section seems to imply that “OAM” is “separate OAM packets”,
where there is OAM Performance Management (PM) considerations for in-band
in-data-packets measurement. Those ought to be considered in the encap layer.
Oops. There is an issue with the nesting of the bullets. Should read:
In terms of what we have heard from the various working groups there
seem to be needs to:
o Be able to send out-of-band OAM messages - that potentially should
follow the same path through the network as some flow of data
packets.
* Such OAM messages should not accidentally be decapsulated and
forwarded to the end stations.
o Be able to add OAM information to data packets that are
encapsulated. Discussions have been around:
* Using a bit in the OAM to synchronize sampling of counters
between the encapsulator and decapsulator.
* Optional timestamps, sequence numbers, etc for more detailed
measurements between encapsulator and decapsulator.
o Usable for both proactive monitoring (akin to BFD) and reactive
checks (akin to traceroute to pin-point a failure)
Does that make it more clear?
There can be several ways to prevent OAM packets from accidentally
being forwarded to the end station using:
o A bit in the frame (as in TRILL) indicating OAM
o A next-protocol indication with a designated value for "none" or
"oam”.
CMP: Or both! An OAM bit with an OAM proto allows for max flexibility.
With all the (hardware and software) implementations having to inspect both?
Anyhow, such details can be worked out in the WGs working on the actual
encapsulations.
Thanks again for your comments.
I'll submit -00 with the changes I have so far.
Erik
Hope these help!
— Carlos.
_______________________________________________
rtgwg mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/rtgwg
_______________________________________________
rtgwg mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/rtgwg
