I am responding to Yakov Rekhter, Christopher Morrow and Xu Xiaohu. Short version:
I think it is still early days for routing scaling solutions. LISP has too many fundamental problems to be considered a potentially practical solution. Some of these problems require design changes and others are probably solvable via extensions to the current design. These problems are well documented. Running more experiments, writing interoperable implementations etc. will not help solve these problems. The other core-edge separation schemes (APT, Ivip, TRRP and Six/One Router) need a lot more development before a sufficient number of people in this field would agree that it was a (or "the") promising solution. Whether in a WG or independent of the IETF, I think each proposal should be developed vigorously - but with all developers taking a keen interest in the other proposals and responding fully in a public discussion list (their own, or the RRG) to those who make constructive critiques of their proposals. Yakov Rekhter wrote: > In answering this question we need to keep in mind that such > techniques as (a) caching routing and forwarding information, (b) > use of separate mapping system for Loc/ID mapping, (c) relying on > probing for determining path feasibility are essential/fundamental > to LISP. In other words, these techniques form the foundation of > LISP. Concerns with scaling and operational properties of these > techniques have been raised many times before (both at the previous > BOF, as well as on the RRG mailing list). Yet the LISP proponents > still did not adequately address these concerns. Some LISP problems which I think are unresolved include the following, many of which are mentioned in the critiques section of: http://www.firstpr.com.au/ip/ivip/lisp-links/ 1 - Delays in delivering initial packets in a flow. This is either due to sending the packets along the ALT network (which takes time and involves sending substantial volumes of data packets over the ALT network, rather than just mapping requests) or to sending mapping requests only, and waiting for the ITR to get a response before it attempts to send traffic packets. According to: http://www.lisp4.net/docs/lisp-ausnog02.ppt pp 10 & 11 the experimental LISP ALT network's ITRs drop initial packets and send brief map request messages. Even if we think this delay doesn't matter, I am sure enough potential adopters would - and therefore make it difficult or impossible for LISP or any other such solution to be widely enough adopted to solve the routing scaling problem. [Fix? This seems to be inherent in the ALT architecture. There's no obvious fix by caching in the ALT network and with millions of nodes (ITRs and ETRs) in the ALT network, there are going to be multiple hops in the network which result in inherent delays which would affect all session establishment, and even single byte communications such as ping, some DNS queries and responses etc.] {Experiments? The problem won't show up in any experimental network - only in real use with tens or hundreds of thousands of ALT routers.} 2 - LISP-ALT's long-path problem http://psg.com/lists/rrg/2008/msg01676.html http://www.antd.nist.gov/~ksriram/strong_aggregation.png [Fix? Another fundamental problem in the architecture. Could be partially solved by more meshiness, but that would greatly increase the complexity of the network and so raise more scaling problems.] {Experiments? As for 1 above.} 3 - Problems creating a highly aggregated ALT network in order to speed the flow of packets up and down the hierarchy, while also making the network robust against the failure of its routers and tunnels. This has not been discussed much on the RRG, but it is an obvious problem. [Fix? Again, only by making it meshier, which detracts from the ALT network's primary design goal - of being highly aggregated to enhance efficiency and to minimise scaling problems.] {Experiments? As for 1 above.) 4 - LISP-ALT's Aggregation implies provider dependence. This is Christian Vogt's critique: http://psg.com/lists/rrg/2008/msg00259.html [Fix? Seems to be a fundamental problem, but every core-edge separation scheme involves supplying stable EID, micronet etc. address space to end-user networks, so there must be some kind of dependence on some organisation - though not necessarily an ISP.] {Experiments? There's no need for experiment - problems such as this can be clearly foreseen.} 5 - Path MTU Discovery problems. Despite Fred Templin, myself and others discussing the inherent PMTUD problems in any map-encap proposal, there has been nothing from the LISP team to indicate they have a solution. They seem to think there is no problem. [Fix? Add some arrangements such as I propose: http://www.firstpr.com.au/ip/ivip/pmtud-frag/ or maybe like something Fred proposes.] {Experiments? Experiment with such a fix, but the real test only comes with many years of full deployment, with a rich mix of jumboframe compatible edge networks and DFZ paths, and with 1500 byte and slightly smaller MTU links still being used in some or many places.} 6 - Lack of business case for LISP's Proxy Tunnel Routers: http://psg.com/lists/rrg/2008/msg02021.html [Fix: Develop a detailed proposal for how end-users would gain their EID address space in a way in which some organisation would have a financial incentive to build and run a set of PTRs all round the world which would advertise a single prefix in the DFZ which would cover the EID space of many (thousands or more) end-user networks. AFAIK LISP could adopt something similar to Ivip's arrangements: http://psg.com/lists/rrg/2008/msg01158.html ] {Experiments? This is not amenable to experiment - there needs to be a good proposal which can only be tested in the commercial reality of widespread deployment. In other words: there will be no widespread deployment of any proposal until the proposal is found to be attractive by the end-user networks which we need to adopt the scalable routing proposal - essentially all of them. That will only happen if the proposal fully supports multihoming etc. of packets sent by hosts in non-upgraded networks. Ivip's OITRDs and LISP's PTRs are the only ways to do this. (APT has an arrangement with some similar business case difficulties as LISP's and TRRP's solution involves a worrying number of carrots and sticks.) So there needs to be an attractive business case for these OITRDs or PTRs.} 7 - The scaling problems of potentially thousands of ITRs each probing reachability to one ETR, and likewise, one ITR probing reachability to many ETRs - this is one view of the "Locator Path Liveness Problem" of draft-meyer-loc-id-implications-00. http://www.irtf.org/pipermail/rrg/2009-January/000809.html [Fix? The problems of ITRs having to determine reachability and make decisions about which ETR to use seem to be fundamental to LISP, APT, TRRP or to any other core-edge elimination scheme which does not have a real-time mapping distribution system.] {Experiments? Not useful at all - this problem won't show up in an experimental network, but can be clearly foreseen in a full-scale deployment scenario.} > Unless these concerns are adequately addressed, claiming that LISP > is an appropriate solution to the problems discussed at the IAB's > October 2006 Routing and Addressing Workshop is nothing more than > a proof by an emphatic assertion. I agree entirely. I believe the LISP team could have made much better use of the RRG - by participating fully in the debates resulting from these critiques. Experiments won't help solve most of these problems. I am not against experimentation and I think it is great that there is a LISP experimental network. However, I would never have taken a proposal to the point of writing code, running a network and inviting others to write compatible implementations when the proposal had so many fundamental problems. I would have worked first to resolve the problems - and if they could not be resolved, I would have abandoned the proposal. LISP's problems are generally not amenable to solution by experiment. I think that a proposal should proceed to experiment once the fundamental problems (which are not amenable to experimental solution) are either resolved, or well on the way to being resolved. The starting point for solving LISP's problems would be admitting them as problems and then discussing them. draft-meyer-loc-id-implications is a step in the right direction, but the problems of having large numbers of ITRs probing large numbers of ETRs, making their own decisions etc. has been obvious since the inception of LISP and APT two years ago. > Addressing these concerns does *not* require LISP protocol specs. > Addressing these concerns does *not* require experimentation with > LISP protocols either. Addressing these concerns does *not* require > interoperable LISP implementations. Therefore, forming a WG to work > on the design on the LISP base protocol , the LISP+ALT mapping > system, LISP Interworking and LISP multicast", and to "encourage > and support interoperable LISP implementations as well as defining > requirements for alternate mapping systems" (as was proposed in > Dave's e-mail) is *not* going to address these concerns. > > Unless the concerns I mentioned above are adequately addressed, > LISP can not be accepted as a feasible/practical approach towards > routing scalability. I agree entirely. > Therefore, unless and until these concerns are adequately > addressed, it is totally premature to form LISP WG. Is the criteria for forming a WG is that the proposal is either "the" solution to the given problem or one of a few such proposals which show sufficient promise to warrant serious effort to develop into a full set of technical specifications? If so, then I don't think LISP is anywhere near the mark, due to the large number of fundamental problems which are nowhere near being solved. Christopher Morrow wrote: > I'm not sure there have been any other credible solutions aside from > 'map/encap', this seems promising, though certainly it hasn't been > proven out in the way that v4 routing as we know it today has been. I would agree with a reworded version of the above, replacing 'map/encap' with: core-edge separation schemes (all of which - apart from Six/One Router - started as map-encap) Although I haven't read draft-templin-ranger, I think it is fair to say there are no Core-Edge Elimination proposals (inherently host-based, and limited to IPv6) which are developed to the level of the three most developed core-edge separation schemes: LISP (ALT and NERD, which is widely regarded as unscalable) APT Ivip (Six/One Router is IPv6-only, and needs to steal a bit from the IPv6 header - which I think would require DFZ router upgrades, which may well be practical. See my critiques: http://psg.com/lists/rrg/2008/msg02034.html http://psg.com/lists/rrg/2008/msg02218.html ) LISP and APT are purely map-encap. Ivip in the longer term, or ideally from the start, is intended to use modified DFZ (and other) routers so some or all of the ETR address can be encoded into a modified version of the IPv4 or IPv6 header. This removes the encapsulation overhead efficiency problem and more importantly involves no PMTUD problems. Ideally, Ivip wouldn't use map-encap at all, but it could use map-encap initially. > Certainly there are use cases where there will be issues, certainly > there are ways to deal with these use cases. Without some direct > experimentation, thought about the problems and focus, we won't be > able to address these things. (we == community) I disagree. The most pressing problems with LISP can be clearly foreseen and do not need any illumination from experimentation. Most of them need to be solved by changing the architecture, though the PMTUD and PTR business case models could be solved by adding to the current architecture. If they can't be solved clearly by changing the design, then it needs to be recognised that these fundamental problems are a strong argument against adopting any such proposal as the solution to the routing scaling problem. Xu Xiaohu wrote: > Exactly. The to-be-formed WG should not only focus on LISP, but > also focus on other map&encap solutions. I think it is widely agreed that each WG should have a single, concrete, focus. I think the best way forward, in addition to the RRG discussions (which I guess will continue after we make our recommendation at the end of March) is for the most promising proposals to be developed as vigorously as possible, while learning from and cross-fertilising the other proposals. I think each such proposal should have its own, potentially private, mailing list so the people concerned can work together in whatever way suits them best. There is now a LISP-Interest mailing list - but my impression is that LISP development to date has been based on private communications between the largely Cisco-based developers. > Just as Yakov etc. pointed out, there are still some open issues in > LISP need to be answered. If many different map&encaps solutions > can be explored further simultaneously under this WG charter, we > may find they are complementary and can be converged finally to > form a successful short-term solution. I think the best way to develop a proposal is a dedicated project - but one in which the developers take a keen interest in critiques of their work, and in the work of other comparable projects. My impression of LISP development to date is that it has been far too insular - uninterested in critiques or on how other proposals work. I think the developers of APT, TRRP and Six/One Router have been much more responsive to critiques. There's no way of forcing the LISP developers to take a greater interest in these things. What would be the purpose of giving the LISP team their own IETF WG? It doesn't allow them to develop anything they can't develop on their own, with their own private or public mailing list, meetings etc. If creating a LISP WG carries some implication that LISP, or anything resembling it, is "nearly ready for widespread adoption" or similar, then I think it would be a mistake. However, if (like HIP and probably many other protocols IETF WGs eventually produce full, interoperable, specifications for) LISP is regarded as being worth working on for experimental purposes AND it is thought that the best way to do this is to form a WG, then a LISP WG would be a good idea. However, I think it is mistaken to believe that writing code, running an experimental network and writing interoperable implementations will solve LISP's fundamental problems. Ideally, after 2 years of developing various proposals and discussing this stuff on the RAM and then the RRG lists, the RRG would be in a position to say something like: This exact proposal, or an architecture with these principles and characteristics, is both a promising way to solve the routing scaling problem and by far the most promising solution. Therefore, we recommend the IETF form a WG to develop this proposal / architecture into a fully fledged specification, with the intention that it be ready for widespread adoption by ISPs and end-user networks by the middle of the next decade. However, I think the RRG is not ready to do this. Despite this problem having been foreseen since the early 1990s, it is still early days in terms of developing the most promising solution. LISP has too many problems, with no obvious solutions in sight and with an apparent reluctance of the developers to discuss many of these problems in public. APT is the work of a smaller group than LISP. It doesn't have LISP's problem with delaying initial packets. The hybrid push-pull mapping system with local full-database query servers solves the delay problem, removes the need for a global query server system (LISP ALT and TRRP) and uses a slow mapping distribution system which doesn't raise any obvious scaling problems. But without running code and a demo system, would the RRG be able to say for sure this might be a practical solution, or the best possible solution? I think not. Ivip has a smaller team still - so far just me, with the help of Steve Russert. Ivip is more ambitious than LISP and APT in some ways, but simpler in others. It will be two years at least before there is running code (but I hope to have a complete set of I-Ds before March). I like to think the RRG would see the promise of Ivip and recommend it to the IETF . . . but it is early days yet for all these proposals. I think the most likely outcome of these last two years is that we agree that a solution needs to be developed by about 2014 which we can be confident will be very widely adopted for IPv4 - with something cooking for IPv6 too. I think we can rule out host-based core-edge elimination schemes on grounds of excessive complexity in all hosts, excessive management traffic to and from hosts - and also in terms of such changes (host changes, probably including major changes to applications) being at odds with the need to have the solution voluntarily adopted by the great majority of end-user networks. That leaves the core-edge separation schemes. There is a big distinction between those which integrate reachability probing and decision making into the ITRs and the architecture itself (LISP, APT, TRRP and the IPv6-only Six/One Router) and those (only one so far - Ivip) which use a real-time mapping system so the reachability testing and "which ETR to use" decision making is under the direct control of each end-user network, and modularly separated from the scheme itself. There is also a big distinction between those schemes (LISP, APT and TRRP) which would forever be stuck with encapsulation overheads and having to solve the PMTUD problems inherent in map-encap, and those (so far Ivip and Six/One Router) which will get the packets across the core without overheads or PMTUD problems. I don't think any of these are developed enough to warrant the IETF developing just one as "the" solution. - Robin http://www.firstpr.com.au/ip/ivip/ _______________________________________________ rrg mailing list [email protected] http://www.irtf.org/mailman/listinfo/rrg
