I'm looking for feedback from anyone interest but, in particular, I'd like feedback from the following people for varying perspectives: Mark McClain (proposed alternate), John Belamaric (IPAM), Ryan Tidwell (BGP), Neil Jerram (L3 networks), Aaron Rosen (help understand multi-provider networks) and you if you're reading this list of names and thinking "he forgot me!"
We have been struggling to develop a way to model a network which is composed of disjoint L2 networks connected by routers. The intent of this email is to describe the two proposals and request input on the two in attempt to choose a direction forward. But, first: requirements. Requirements: The network should appear to end users as a single network choice. They should not be burdened with choosing between segments. It might interest them that L2 communications may not work between instances on this network but that is all. This has been requested by numerous operators [1][4]. It can be useful for external networks and provider networks. The model needs to be flexible enough to support two distinct types of addresses: 1) address blocks which are statically bound to a single segment and 2) address blocks which are mobile across segments using some sort of dynamic routing capability like BGP or programmatically injecting routes in to the infrastructure's routers with a plugin. Overlay networks are not the answer to this. The goal of this effort is to scale very large networks with many connected ports by doing L3 routing (e.g. to the top of rack) instead of using a large continuous L2 fabric. Also, the operators interested in this work do not want the complexity of overlay networks [4]. Proposal 1: We refined this model [2] at the Neutron mid-cycle a couple of weeks ago. This proposal has already resonated reasonably with operators, especially those from GoDaddy who attended the Neutron sprint. Some key parts of this proposal are: 1. The routed super network is called a front network. The segments are called back(ing) networks. 2. Backing networks are modeled as admin-owned private provider networks but otherwise are full-blown Neutron networks. 3. The front network is marked with a new provider type. 4. A Neutron router is created to link the backing networks with internal ports. It represents the collective routing ability of the underlying infrastructure. 5. Backing networks are associated with a subset of hosts. 6. Ports created on the front network must have a host binding and are actually created on a backing network when all is said and done. They carry the ID of the backing network in the DB. Using Neutron networks to model the segments allows us to fully specify the details of each network using the regular Neutron model. They could be heterogeneous or homogeneous, it doesn't matter. This proposal offers a clear separation between the statically bound and the mobile address blocks by associating the former with the backing networks and the latter with the front network. The mobile addresses are modeled just like floating IPs are today but are implemented by some plugin code (possibly without NAT). This proposal also provides some advantages for integrating dynamic routing. Since each backing network will, by necessity, have a corresponding router in the infrastructure, the relationship between dynamic routing speaker, router, and network is clear in the model: network <-> speaker <-> router. Proposal 2: This alternate model has not been fully fleshed out. Some parts of it are still unclear to me. The basic idea is to give the IPAM system information about IP availability on a given host. When creating a port, the binding information would be sent to the IPAM system and the system would choose an appropriate address block for the allocation. 1. This alternate model offers no way to distinguish the two types of address blocks. 2. We don't have the benefit of modeling the segments with Neutron networks. It was suggested that hierarchical port binding could help here but I see it as orthogonal to this. Hierarchical port binding extends the L2 properties of a port to a hierarchical infrastructure to achieve continuous L2 connectivity. It is also intended for overlay networks. That isn't what we're doing here and I don't think it fits. I have also considered the multi-provider extension [3] for this. This is not yet clear to me either. First, my understanding was that this extension describes multi-segment continuous L2 fabrics. Second, there doesn't seem to be any host binding aspect to the multi-provider extension. Third, not all L2 plugins support this extension. It seems silly to require L2 plugin support in order to enable routing between segments. It isn't clear to me how a dynamic routing speaker will fit in to this model. My first thought is that it must be integrated with IPAM because the IPAM system has the understanding of how to map address blocks to infrastructure. This pushes even more infrastructure knowledge down to the IPAM system. If dynamic routing is pushed down to the IPAM system, it will also be necessary to push the association of mobile IPs or routed tenant subnets down in to the IPAM system too. This means Neutron needs to tell IPAM about every floating IP association and every tenant subnet behind a Neutron router in the same address scope as the external network. I'm not convinced that IPAM and routing really belong together like this. If you made it this far in this email, you must have some feedback. Please help us out. Carl Baldwin [1] https://bugs.launchpad.net/neutron/+bug/1458890 [2] https://review.openstack.org/#/c/196812/ [3] http://developer.openstack.org/api-ref-networking-v2-ext.html#network_multi_provider-ext [4] https://etherpad.openstack.org/p/Network_Segmentation_Usecases __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
