Short version: The Devil is in the detail, but the I-Ds don't yet
have all the details. Multiple levels of caching
is good in some ways and troublesome in others.
To: RRG, CC: LISP list.
I suggest discussing this on the RRG list. I figure
all people are on the RRG list, but not vice-versa.
The questions of multiple points of caching are of
general architectural interest regarding potential
scalable routing solutions other than LISP.
There is a new LISP I-D (2009-03-03):
http://tools.ietf.org/html/draft-fuller-lisp-ms-00
and an updated one (2009-03-02) to match:
http://tools.ietf.org/html/draft-farinacci-lisp-12
LISP Map Server draft-fuller-lisp-ms-00
Abstract
This draft describes the LISP Map-Server (LISP-MS), a computing
system which provides a simple LISP protocol interface as a "front
end" to the Endpoint-ID (EID) to Routing Locator (RLOC) mapping
database and associated virtual network of LISP protocol elements.
The purpose of the Map-Server is to simplify the implementation
and operation of LISP Ingress Tunnel Routers (ITRs) and Egress
Tunnel Routers (ETRs), the devices that implement the "edge" of the
LISP infrastructure and which connect directly to LISP-capable
Internet end sites.
My understanding of and comments on this are:
Instead of ITRs and ETRs needing to act as routers in the ALT
network, they communicate via the ordinary Internet with Map Servers,
which are routers on the ALT network. This will greatly reduce the
complexity and configuration difficulties of ITR and ETRs.
These Map Server devices are implicitly local to the ITRs and ETRs in
a given network and are intended to be used only by those ITRs and
ETRs. They are always on RLOC (stable, globally reachable, non
LISP-mapped) addresses.
There are two functions which may be combined in the one device:
Map Resolver (MR)
Accepts a mapping query from an ITR and (usually) sends the
ITR a mapping reply. (The exception is if the MR doesn't
have the information and sends the query verbatim to some
other device, which will answer the query directly to the
ITR.)
MRs can be caching or non-caching. More on that below.
ITRs are intended to be configured with a single address for
their local MR. This would raise questions of robustness if
not for the next item:
Multiple MRs in a local network (such as an ISP network or
I guess any end-user network which has ITRs) can be configured
on the one anycast address. This way, the ITR's request will
be forwarded to the nearest currently active MR. All
communication is via single packets, not via TCP. Presumably
the MRs will also have their own unique addresses so they can
be managed via TCP.
I think the MR is an important improvement to LISP-ALT, since
it enables an ITR to be a much more casual and unstable concept
than was the case when all ITRs needed to participate in the
ALT network as routers (AFAIK). This means that ITRs can be
added easily, without having to configure anything.
It also means (though this is my suggestion, not from the LISP
team) that an ITR function could easily be implemented in a
sending host, assuming it was not behind NAT. I guess the
sending host would need to be on an RLOC address - which rules
out this idea for sending hosts in end-user networks. Ivip's
ITR in sending host function (ITFH) requires the host to be
on a non-NAT address which can be and ordinary or a Scalable PI
address - RLOC or EID in LISP parlance.
Map Server (MS)
Is a router on the ALT network and accepts secure messages from
one or more ETRs. (Secret key pairs to secure these.) ETRs
are (typically, or always?) the authoritative source of mapping
information in LISP.
ETRs can be on any RLOC address and use ordinary packets to
communicate with the MS.
My understanding is that the MS announces the appropriate
prefixes on the ALT network - one for every EID the ETR
tells it.
Ignoring MSes for a moment, I have never understood how this
would work with two ETRs in two separate ISPs handling the same
EID. Both ETRs would be routers on the ALT network and would
announce the same prefix. So where do packets go to? I guess
to either. But then the ETRs somehow need to coordinate
themselves, or be coordinated by something else, so they act
in a unified manner. Then, as long as both were reachable and
working properly, it wouldn't matter which ETR got the query.
The same problem seems to apply with MSes. There would be two
ETRs in two separate ISPs and each would presumably (for
robustness in a multihoming situation and probably for security
reasons) have its own MS in its own ISP network.
So now we have two ETRs and two MSes which need to be
coordinated. The two MSes both announce the one EID prefix
on the ALT network. Yet they are supposed to still be
coordinated during outages.
However this is resolved, I think it is a big improvement for
LISP to have MSes, since it reduces the cost, complexity,
management effort etc. for ETRs similarly to how MRs do the
same for ITRs.
Both these functions can presumably be performed quite adequately by
software devices, such as a COTS server with suitable software.
There doesn't have to be any hardware router FIB etc. AFAIK.
This would enable hardware routers to assume ITR and ETR
responsibilities without them also needing all the software and
configuration, stable address etc. to be an ALT router. Also, by
decreasing the total number of ALT routers, this simplifies the ALT
network.
I gather from this new I-D, and from what I read in:
http://www.lisp4.net/docs/lisp-ausnog02.ppt
that the current test network and the intention for the future is not
to send traffic packets on the ALT network. This approach was
initially an option, with the intention that the ALT network would
forward the initial packet(s) to the correct ETR, which would then
forward it to the destination network, while also recognising it as a
map request and so would send a map reply message to the ITR.
I recall from somewhere that the ITR typically sends out a few
mapping requests, just in case one of them is dropped. When the ITR
connects directly to the ALT network, these packets presumably
usually traverse the entire global ALT network until they are
delivered to one or more (probably just all to one) ETR which
responds. I guess the ETR sends multiple replies, but maybe not.
The reply goes to the ITR via the ordinary Internet.
Removing these potentially long and voluminous traffic packets from
the ALT network seems like a good idea to me. There may well be
security benefits in doing so too. Below, I assume the ALT network
only carries mapping requests, and that the map replies go back from
whatever answers them (an ETR connected to ALT network, or more
likely a Map Server) via a direct ordinary Internet packet to the
device which made the query (perhaps a directly connected ITR or more
likely a Map Resolver).
A Caching Map Resolver?
If the MR caches, then it has the potential to significantly reduce
the traffic on the ALT network. This is due to two or more ITRs in a
given ISP network wanting the same mapping, and the second and
subsequent ones getting it directly from the local caching MR.
This also has the potential to eliminate, for the second and
subsequent ITRs which need this mapping, the major problem of
"LISP-ALT's initial packet delays", so much debated on the RRG in
recent months.
There is nothing in draft-fuller-lisp-ms-00 to describe this caching
behavior.
The caching time of map replies is specified in units of one minute:
draft-farinacci-lisp-12:
Record TTL: The time in minutes the recipient of the Map-Reply
will store the mapping.
Let's say at time T = 0 minutes, ITR-A sends a map request to MR-1,
which has no mapping for the EID prefix which matches the EID address
in the request message. MR-1 sends its own map request message (with
its own nonce) onto the ALT network which forwards it to either the
single Map Server which advertises the matching EID prefix on the ALT
network, or to one of the multiple such Map Serves, or perhaps to the
directly ALT-connected ETR(s) which do the same.
That device sends the mapping reply back to MR-1 directly via the
Internet. The reply is secured by returning MR-1's nonce.
Let's say the mapping reply comes back with a 90 minute caching time.
MR-1 sends to ITR-A a map reply, with ITR-A's request's nonce, with
the fresh mapping information and a caching time of 90 minutes. Now
MR-1 can encapsulate packets to its choice of ETRs, based on the
fresh mapping it has received and whatever it has determined about
reachability of those ETRs, and of the ETRs' ability to get packets
to the destination network.
Later, at T = 85 minutes, ITR-B sends a mapping request to MR-1 for
an address which matches this same EID prefix. MR-1 can use its
cached information and send a reply within a few milliseconds. This
means ITR-B's traffic will not be delayed by any significant amount.
What caching time will be in that reply to ITR-B? I assume it will
be 5 minutes. If it would be 90 minutes, ITR-B could be running for
a long time to come on stale mapping information.
Assuming ITR-A no longer needs this EID's mapping, but ITR-B keeps
needing to tunnel packets addressed to this EID, then at T=90
minutes, ITR-B will want mapping information again.
Should ITR-B request the mapping again at at T = 88 minutes, in
readiness for probably needing it in 1 minute's time?
This would seem like a generally reasonable approach if it prompted
MR-1 to get fresh mapping information, but why should MR-1 do this?
Would MR-1 need to look at the original caching time and how much
has expired to decide whether it should, by some algorithm, request
fresh mapping? But what if the mapping hadn't changed in the distant
Map Server, but the ETR was going to change it two minutes later?
If ITR-B waited until T = 90 or a little later before requesting
fresh mapping, then unless MR-1 had already got fresh mapping in the
last minute or two, then there would presumably be a delay in the ITR
being able to handle traffic for this EID, since it would take some
time for MR-1's second mapping request to traverse the ALT network
and generate a reply to MR-1.
There are various scenarios, but I think there are potential
difficulties with caching times running out in three locations now
rather than one.
Previously, it was simple (despite the scaling problems of lots of
ITRs peppering an ETR for mapping, not to mention them all trying to
decide reachability for this and other ETRs):
ITR query -----------> ETR
<----------- reply
cache
Now we have:
ITR query -----> Map query -----> Map
<----- reply Resolver <----- reply Server <--Register- ETR
Cache Cache Cached, in a sense,
controlled by messages
from the ETR(s) whenever
they can reach the
Map Server and decide
to send a Map Register
message.
I think this raises more complex problems with:
1 - How to avoid cache times running out at ITRs
which are going to be tunneling packets addressed
to this EID after the cache time expires.
Such a situation will cause a traffic delay unless
the local Map Resolver has recently got fresh mapping.
2 - How to minimise unnecessary map requests by Map
Resolvers trying to anticipate ITRs making such
requests, but actually requesting fresh mapping from
the distant Map Server when the ITR doesn't need it.
Without further complications, the Map Resolver can't know whether
the one or more ITRs which requested the mapping for an EID are still
handling traffic for that EID. So it can't very well request fresh
mapping towards the end of its expiry, just in case an ITR wants it.
To do so would approximately double the volume of map requests
traversing the ALT network, since it is reasonable to assume, with
longish caching times, that the original caching time will generally
suffice for the needs of the one or more ITRs served by the Map
Resolver. (This would not be true with a busy Map Resolver and
popular EIDs many ITRs are sending packets to.)
Without some elaboration of the request protocol, ITR-B at T = 85
minutes can't ask the Map Resolver to get fresh mapping and send it a
new reply - unless there is some algorithm in the Map Resolver such
as: "If the cached mapping is 90% of the way to its expiry time, do
not answer the new request from the cache, but send a fresh map
request and then answer the query if and when the new reply arrives."
To do so would effectively shorten all the caching times.
At present, there is only one kind of map request message from an ITR
to a Map Server - implicitly an urgent request.
If there was a second kind:
"This ITR has mapping for this EID which will expire in some
time period (specified) soon, and requests the Map Resolver
to get fresh mapping from the Map Server now, and to send
a reply once this arrives."
then I think these problems would be resolvable with less trouble and
less need for choices based on limited information.
- Robin
_______________________________________________
rrg mailing list
[email protected]
http://www.irtf.org/mailman/listinfo/rrg
