Hello Wes,
I've read this draft. I'll note that I'm already in the
acknowledgements section, but I can't find an on-list review that I
wrote on this draft prior to this point unless I sent it privately to
the authors, in which case I apologize if I'm retreading old
discussion.
That was based on your review and off-list mail of June 23rd 2010.
A nit regarding the acknowledgement - my first name is
Wes, not George.
Apologies for that. Will be fixed.
Some Nits Abstract: %s/build/built in parallel, /co-exit/co-exist Now
that the draft is a WG doc, "..the authors believe..." (last
sentence) can probably be removed (from intro section as well).
Intro: "other then best path" - suggest rewording to something like
"secondary and tertiary paths" or "alternate paths which are not
considered best path"
2. %s/"reduction of time of reachability restoration"/faster
reachability restoration
Will fix.
Other stuff: 2.1 - when discussing overhead and scale concerns for
add paths, perhaps a citation to 4984 would be appropriate?
I would prefer not to mix the growing internet scale concerns from some
of the operational practices/configuration based based scale concerns.
I've made
similar comments to the SIDR folks, and I think generally anything
that adds a non-trivial amount of impact to the growth curve of the
routing system needs to consider this.
I think there is substantial difference for local vs global size
increase of the routing system. Here in this work all concerns are
regarding to the local one.
Also, why is this citing
implementation data from 2009? Is there now an implementation that
supports add-paths, or is this just a holdover from earlier versions
of the draft? If there is now an implementation, it would be good to
revisit this section.
The point to cite this was to compare with original 2002 time when
add-paths was defined to indicate how long it took for implementations
to finally start looking at this. I will reword this section to better
reflect the intention and avoid using absolute dates.
4. This asserts that no code changes are necessary to RR clients. I'm
not sure I totally agree with that... If the idea is to have a
primary (best) RR and then N additional paths, the general assumption
is that the N, N1, ... RRs are carrying routes that are less and less
preferred. How does this system avoid the same sort of inconsistency
of best path choice among different routers in the network if there
is no way to identify those paths as secondary? I think you need some
way to determine if the alternate routes are intended to be ECMP
routes or backup routes... You may be able to cover this without code
changes by using alternate configurations of other BGP preference
indicators (MED, Localpref, metric, etc), perhaps with inbound route
policy on the client or outbound on the RR, but since things like
metric may be different based on where something is in the network,
that may lead to inconsistency if used by itself. Even then, the
draft doesn't discuss how this should be managed.
I stand by the claim that no code change is needed on clients. Moreover
no even additional policy change is required either.
The best way to illustrate this is to compare presence of additional BGP
paths on the clients in the scenario where clients are interconnected
with full IBGP mesh or would get all paths with add-path. In neither
case there is a notion of RR telling client which path is best or which
is second best .. and there is number of good reasons for that (one is
that for RR numbering paths can be different then for client, the other
one is that when we would withdraw any path advertised and ordered we
would need to re advertise with new order all remaining paths - that
amount of churn is non negligible).
Each client's BGP best path is capable of making safe (loop free)
autonomous choice of paths in PIC/fast connectivity restoration/ibgp
multipath cases.
4.1 Also, there's a definite scaling consideration on the RR clients
that isn't really discussed here - they are now going to be storing
some number of additional routes and paths that is linearly related
to the number of additional planes that are implemented. The addition
of more RR sessions that presumably carry a portion of the full
routing table now drives a non-trivial increase in memory footprint
and processing overhead (and potentially convergence time for slower
boxes). In the simplest case of 2 primary route-reflectors (for
diversity), and 1 2nd-best path RR, you've added one session. If you
want to carry a 3rd-best RR or have redundant 2nd-best RRs, you've
added 4 sessions. It's fair to say that after a certain number of
alternate paths, you start having less routes because there are only
so many alternative exits, but otherwise there is a potentially large
problem even if it's not quite as bad as addpaths. I might recommend
that you do some analysis of the routing table to know where this
threshold makes a difference, based on how many alternate paths an
average route carries. In addition to being a scaling consideration,
it also helps to inform what value of N becomes diminishing returns
because most networks don't have that many backup paths. I envision
this being something like "80% of routes have 4 or less paths, so
moving beyond 4 planes may add overhead without much benefit..."
It is absolutely correct to say that more paths client carries the more
CPU cycles and memory will be used to process and store them.
However there is one observation to be made ... in 99% of cases I have
seen for distributing more then best path intra-domain the sufficient
number of paths per net on each client is 2.
Moreover this is configurable by the operator modulo RR support
capabilities for more planes then 2.
IMHO cost of bringing additional paths for control plane is quite well
understood today. Moreover it is quite implementation dependent. Some
implementation may use X bytes per path while the other one Y bytes to
store the same path. I think some separate BGP scaling document (even as
BCP) may be equally useful for any technique to advertise more then best
path. I would prefer to keep this outside of the solutions work on how
to advertise and distribute those additional paths.
Further scaling considerations occur in the core if the P routers
must know about both the primary and secondary paths (and therefore
peer/mesh with both R1 and R1'). You may be able to draw the
conclusion that this is better than full mesh (5.1 #3) and therefore
superior from a scaling perspective, but this at least needs to be
discussed. It may be that the only way to avoid that particular issue
is to operate with a BGP-free core...
>
> It may be appropriate to add a separate scaling considerations
> discussion to your deployment considerations (section 6) to discuss
> some of the above.
I agree 100% .. but as stated above I do not find this specific to
diverse-path. It seems a general issue and I would highly encourage
someone to take a stub to document this in IETF/IDR/GROW or maybe at
Nanog community repository.
There may be additional operational considerations from the
perspective of route analysis - if you have either a homebuilt or off
the shelf set of software that does route analysis for the purpose of
event root-cause analysis, anomaly detection, capacity
planning/failure analysis, etc, it has to be aware of these
additional planes such that it returns the proper response when
evaluating the routing table to determine what the expected behavior
should be in the real network. This is especially important when it
uses the table to determine how traffic will reroute during different
failure scenarios. These tools may act like a participant in the mesh
rather than a client in order to get a pure view of the table, and
that may lead to undesired results if the multiple planes aren't
taken into account. There may also be considerations for looking
glass implementations and the actual information that is visible on
the RRs and RR clients as the result of standard BGP show commands
to aid in troubleshooting and verification.
Very good point. Two comments on this ..
- As to the impact to the tools I am less worried as presence of
additional paths can be a fact today as already mentioned with full mesh
or as used by some operator's by playing with adjusting different weight
values of pair of RRs on a per net basis.
- The use of "planes" in the draft is more of a conceptual nature. In
practice all paths are still kept in the single table where normal best
path is calculated. That means that tools like looking glass should not
observe any changes nor impact.
Once more time many thx for your review.
Robert.
_______________________________________________
GROW mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/grow
