Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On 3/22/15, 8:05 PM, Ian Wells ijw.ubu...@cack.org.ukmailto:ijw.ubu...@cack.org.uk wrote: Seems to me that an address pool corresponds to a network area that you can route across (because routing only works over a network with unique addresses and that's what an address pool does for you). We have those areas and we use NAT to separate them (setting aside the occasional isolated network area with no external connections). But NAT doesn't separate tenants, it separates externally connected routers: one tenant can have many of those routers, or one router can be connected to networks in both tenants. We just happen to frequently use the one external router per tenant model, which is why address pools *appear* to be one per tenant. I think, more accurately, an external router should be given an address pool, and tenants have nothing to do with it. I think conflating address pools with routable space is a mistake. To me, this is the concept of address scope which I see as distinct from pool. For example, a single shared routable space may have several pools, each a /8 which is owned by a specific tenant. This is something that I would like to see in Liberty, making a the concept of an address scope a first class concept. Routers would be able to attach only to networks within the same scope, unless NAT was applied. John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On Sun, Mar 22, 2015 at 05:05:17PM -0700, Ian Wells wrote: On 22 March 2015 at 07:48, Jay Pipes jaypi...@gmail.com wrote: On 03/20/2015 05:16 PM, Kevin Benton wrote: To clarify a bit, we obviously divide lots of things by tenant (quotas, network listing, etc). The difference is that we have nothing right now that has to be unique within a tenant. Are there objects that are uniquely scoped to a tenant in Nova/Glance/etc? Yes. Virtually everything is :) Everything is owned by a tenant. Very few things are one per tenant, where is where this feels like it's leading. Ah, sorry, yes, I misunderstood Kevin's implication there. That is correct. Security group names are, AFAIK, the only thing in Nova that is unique within a tenant. All other resources are identified via UUID, and are not unique within a tenant (project). Seems to me that an address pool corresponds to a network area that you can route across (because routing only works over a network with unique addresses and that's what an address pool does for you). We have those areas and we use NAT to separate them (setting aside the occasional isolated network area with no external connections). But NAT doesn't separate tenants, it separates externally connected routers: one tenant can have many of those routers, or one router can be connected to networks in both tenants. We just happen to frequently use the one external router per tenant model, which is why address pools *appear* to be one per tenant. I think, more accurately, an external router should be given an address pool, and tenants have nothing to do with it. Gotcha. Yep, that makes total sense. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
I think that moving the discussion in whether a pool represents a tenant's routable address space, or whether we need a new (another?!) API entity do deal with it probably does not really fall within the scope of this thread. I am pretty sure Carl will soon push a specification for address scope management - and then gerrit would be the place to debate the details. Neutron does not refuse the idea of using tenants as a logical unit of isolation. However the tenant is not the only way of isolating resources. In this thread we saw people quoting examples where within the same tenant several isolated (from a network perspective) environments are implemented. This are usually L3 domains made by one or more L2 segments connected to a logical router; external network connectivity might or might not be achieve using NAT. I think the goal of subnet pools is to use these environments as units of isolations and ensure no overlapping CIDRs there. However, since there is no way to identify such environments at the API layers, API clients will need to be diligent and follow a workflow where they create a subnetpool, and then for every subnet in a given L3 domain they alway allocate from the same pool. This is probably not ideal from a usability perspective, but anyway enables a new use case. Since (in my opinion) in the vast majority each tenant only deploy one of such L3 domains, having a 1:1 association between a tenant and subnet pool might be a decent idea. However, this will break the use cases previously presented in this thread. The Neutron API, like all OpenStack APIs, has the sweet burden of having to being extremely flexible: you need to able to implement something - and the also the exact opposite. What becomes hard here is to ensure the API stays flexible and usable at the same time. I've thought a bit about this and I have some ideas, obviously with no pretence to translate any of them in code for the Kilo release: 1- each tenant has a default subnet pool, just like each tenant has a default security group. Subnets are by default always allocated by the pool, unless the user explicitly decides not to (e.g.: by explicitly passing null to the subnet_pool attribute - this is just an example based on the merged API don't take it as a proposal!!!) - or unless another pool is explicitly specified. A user indeed can then create additional subnet pools for its tenant; shared pools set up by the cloud operators can also be used. This will enable users to automatically leverage subnetpools. For deployments were multiple L3 isolation environments exist, this will however represent a backward incompatible change. Indeed client apps will have to change there as they will either need to make explicit use of pools or explicitly disable usage of a subnet pool. However, how prefixes for the default subnet pool should be selected is a kind of open question. Perhaps this could be sorted at configuration level. 2- Subnet pools are read only for all tenants. The operator will then decide whether per-tenant pools should be enabled, and decide what the prefixes for the default subnet pool should be.The operator will also set up one or more shared pools. A tenant will be able to read details of such pools (its own default pool and the shared ones), and allocate from them, but won't be able to make any change. Tenants won't be able to create new pools. This is probably simple, but has two drawbacks. The first, and possible less important, is that it won't be possible to implement use cases with multiple domains per tenant in pool-enabled deployments. As a corollary, the second issue - a show stopper in my opinion - is that the Neutron API will become even more dependent on the deployment. I therefore do not think this is an option we really want to pursue. I realize the ideas listed above are probably terrible. I am sharing them indeed in the hope that they will induce somebody else to come up with better ideas ;) Salvatore On 23 March 2015 at 16:41, Jay Pipes jaypi...@gmail.com wrote: On Sun, Mar 22, 2015 at 05:05:17PM -0700, Ian Wells wrote: On 22 March 2015 at 07:48, Jay Pipes jaypi...@gmail.com wrote: On 03/20/2015 05:16 PM, Kevin Benton wrote: To clarify a bit, we obviously divide lots of things by tenant (quotas, network listing, etc). The difference is that we have nothing right now that has to be unique within a tenant. Are there objects that are uniquely scoped to a tenant in Nova/Glance/etc? Yes. Virtually everything is :) Everything is owned by a tenant. Very few things are one per tenant, where is where this feels like it's leading. Ah, sorry, yes, I misunderstood Kevin's implication there. That is correct. Security group names are, AFAIK, the only thing in Nova that is unique within a tenant. All other resources are identified via UUID, and are not unique within a tenant (project). Seems to me that an address pool corresponds to a network area that
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On Mon, Mar 23, 2015 at 9:52 AM, Salvatore Orlando sorla...@nicira.com wrote: I think the goal of subnet pools is to use these environments as units of isolations and ensure no overlapping CIDRs there. However, since there is no way to identify such environments at the API layers, API clients will need to be diligent and follow a workflow where they create a subnetpool, and then for every subnet in a given L3 domain they alway allocate from the same pool. This is probably not ideal from a usability perspective, but anyway enables a new use case. Since (in my opinion) in the vast majority each tenant only deploy one of such L3 domains, having a 1:1 association between a tenant and subnet pool might be a decent idea. However, this will break the use cases previously presented in this thread. +1 Well said. Incidentally, Ryan has a proposal [1] now to constrain new subnet pools to not allow overlapping addresses. I think I'd like to curb the practice of creating new subnet pools that allow overlap before it starts. This does not affect the null pools which will stay backwards compatible forever. But, I tend to agree with Salvatore that the very concept of a pool is at odds with overlapping IPs. The Neutron API, like all OpenStack APIs, has the sweet burden of having to being extremely flexible: you need to able to implement something - and the also the exact opposite. What becomes hard here is to ensure the API stays flexible and usable at the same time. I've thought a bit about this and I have some ideas, obviously with no pretence to translate any of them in code for the Kilo release: 1- each tenant has a default subnet pool, just like each tenant has a default security group. Subnets are by default always allocated by the pool, unless the user explicitly decides not to (e.g.: by explicitly passing null to the subnet_pool attribute - this is just an example based on the merged API don't take it as a proposal!!!) - or unless another pool is explicitly specified. A user indeed can then create additional subnet pools for its tenant; shared pools set up by the cloud operators can also be used. The above sounds like what we'll have for Kilo except that null is the only default available at the moment. We had talked about having a global default pool but not one per-tenant. I think I'd like the global default (configurable in neutron.conf?) for the use case where an operator wants a single routable ipv6 space for an entire cloud as the minimum for Kilo. There should be an independent default for each address family. I'm not sure if the default should preclude the use of the legacy null pool or not. I'll propose this patch and we can discuss there. This will enable users to automatically leverage subnetpools. For deployments were multiple L3 isolation environments exist, this will however represent a backward incompatible change. Indeed client apps will have to change there as they will either need to make explicit use of pools or explicitly disable usage of a subnet pool. However, how prefixes for the default subnet pool should be selected is a kind of open question. Perhaps this could be sorted at configuration level. This needs more discussion and thought. I'm not sure what you mean about how prefixes for the default subnet pool should be selected. 2- Subnet pools are read only for all tenants. The operator will then decide whether per-tenant pools should be enabled, and decide what the prefixes for the default subnet pool should be.The operator will also set up one or more shared pools. A tenant will be able to read details of such pools (its own default pool and the shared ones), and allocate from them, but won't be able to make any change. Tenants won't be able to create new pools. Interesting. Why disable pools for tenants? This is probably simple, but has two drawbacks. The first, and possible less important, is that it won't be possible to implement use cases with multiple domains per tenant in pool-enabled deployments. As a corollary, the second issue - a show stopper in my opinion - is that the Neutron API will become even more dependent on the deployment. I therefore do not think this is an option we really want to pursue. I realize the ideas listed above are probably terrible. I am sharing them indeed in the hope that they will induce somebody else to come up with better ideas ;) Carl [1] https://review.openstack.org/#/c/166397/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On 03/20/2015 05:16 PM, Kevin Benton wrote: To clarify a bit, we obviously divide lots of things by tenant (quotas, network listing, etc). The difference is that we have nothing right now that has to be unique within a tenant. Are there objects that are uniquely scoped to a tenant in Nova/Glance/etc? Yes. Virtually everything is :) Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On 22 March 2015 at 07:48, Jay Pipes jaypi...@gmail.com wrote: On 03/20/2015 05:16 PM, Kevin Benton wrote: To clarify a bit, we obviously divide lots of things by tenant (quotas, network listing, etc). The difference is that we have nothing right now that has to be unique within a tenant. Are there objects that are uniquely scoped to a tenant in Nova/Glance/etc? Yes. Virtually everything is :) Everything is owned by a tenant. Very few things are one per tenant, where is where this feels like it's leading. Seems to me that an address pool corresponds to a network area that you can route across (because routing only works over a network with unique addresses and that's what an address pool does for you). We have those areas and we use NAT to separate them (setting aside the occasional isolated network area with no external connections). But NAT doesn't separate tenants, it separates externally connected routers: one tenant can have many of those routers, or one router can be connected to networks in both tenants. We just happen to frequently use the one external router per tenant model, which is why address pools *appear* to be one per tenant. I think, more accurately, an external router should be given an address pool, and tenants have nothing to do with it. -- Ian. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On 03/11/2015 06:48 PM, John Belamaric wrote: This has been settled and we're not moving forward with it for Kilo. I agree tenants are an administrative concept, not a networking one so using them for uniqueness doesn't really make sense. In Liberty we are proposing a new grouping mechanism, as you call it, specifically for the purpose of defining uniqueness - address scopes. This would be owned by a tenant but could be shared across tenants. It's still in the early stages of definition though, and more discussion is needed but should probably wait until after Kilo is out! This is a question purely out of curiousity. Why is Neutron averse to the concept of using tenants as natural ways of dividing up the cloud -- which at its core means multi-tenant, on-demand computing and networking? Is this just due to a lack of traditional use of the term in networking literature? Or is this something more deep-grained (architecturally) than that? Genuinely curious. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On Fri, Mar 20, 2015 at 12:31 PM, Jay Pipes jaypi...@gmail.com wrote: This is a question purely out of curiousity. Why is Neutron averse to the concept of using tenants as natural ways of dividing up the cloud -- which at its core means multi-tenant, on-demand computing and networking? From what I've heard others say both in this thread and privately to me, there are already a lot of cases where a tenant will use the same address range to stamp out identical topologies. It occurred to me that we might even being doing this with our own gate infrastructure but I don't know for sure. Is this just due to a lack of traditional use of the term in networking literature? Or is this something more deep-grained (architecturally) than that? We already have NAT serving as the natural divider between them and so there is no reason to create another artificial way of dividing them up which will force them to change their practices. I've come to terms with this since my earlier replies to this thread. Carl __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On 03/20/2015 03:37 PM, Carl Baldwin wrote: On Fri, Mar 20, 2015 at 12:31 PM, Jay Pipes jaypi...@gmail.com wrote: This is a question purely out of curiousity. Why is Neutron averse to the concept of using tenants as natural ways of dividing up the cloud -- which at its core means multi-tenant, on-demand computing and networking? From what I've heard others say both in this thread and privately to me, there are already a lot of cases where a tenant will use the same address range to stamp out identical topologies. It occurred to me that we might even being doing this with our own gate infrastructure but I don't know for sure. Is this just due to a lack of traditional use of the term in networking literature? Or is this something more deep-grained (architecturally) than that? We already have NAT serving as the natural divider between them and so there is no reason to create another artificial way of dividing them up which will force them to change their practices. I've come to terms with this since my earlier replies to this thread. OK, thanks for the info, Carl, appreciated. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On 2015-03-20 13:37:49 -0600 (-0600), Carl Baldwin wrote: From what I've heard others say both in this thread and privately to me, there are already a lot of cases where a tenant will use the same address range to stamp out identical topologies. It occurred to me that we might even being doing this with our own gate infrastructure but I don't know for sure. [...] We don't really. I mean we do reuse identical network configurations for jobs running in parallel but these exist solely within the nested OpenStack environments (either entirely local to a worker or in some cases bridged through a GRE tunnel between workers). Now that's for the test infrastructure specifically... if the networks for a Tempest-created tenant in DevStack on one of those workers duplicates a CIDR I wouldn't necessarily know, since I don't pay particularly close attention to DevStack/Tempest configurations. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
To clarify a bit, we obviously divide lots of things by tenant (quotas, network listing, etc). The difference is that we have nothing right now that has to be unique within a tenant. Are there objects that are uniquely scoped to a tenant in Nova/Glance/etc? On Fri, Mar 20, 2015 at 12:50 PM, Jay Pipes jaypi...@gmail.com wrote: On 03/20/2015 03:37 PM, Carl Baldwin wrote: On Fri, Mar 20, 2015 at 12:31 PM, Jay Pipes jaypi...@gmail.com wrote: This is a question purely out of curiousity. Why is Neutron averse to the concept of using tenants as natural ways of dividing up the cloud -- which at its core means multi-tenant, on-demand computing and networking? From what I've heard others say both in this thread and privately to me, there are already a lot of cases where a tenant will use the same address range to stamp out identical topologies. It occurred to me that we might even being doing this with our own gate infrastructure but I don't know for sure. Is this just due to a lack of traditional use of the term in networking literature? Or is this something more deep-grained (architecturally) than that? We already have NAT serving as the natural divider between them and so there is no reason to create another artificial way of dividing them up which will force them to change their practices. I've come to terms with this since my earlier replies to this thread. OK, thanks for the info, Carl, appreciated. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On Tue, Mar 10, 2015 at 12:06 PM, Ryan Moats rmo...@us.ibm.com wrote: While I'd personally like to see this be restricted (Carl's position), I know of at least one existence proof where management applications are doing precisely what Gabriel is suggesting - reusing the same address range to minimize the configuration differences. Okay, I see the problem. I'm not particularly thrilled about this situation but I see the problem. Can we deprecate this mis-feature? Is there hope that we can stop this madness some day? :) Regardless, it sounds like we need to support this for at least some time in the future. So, here is what I propose: When talking with external IPAM to get a subnet, Neutron will pass both the cidr as the primary identifier and the subnet_id as an alternate identifier. External systems that do not allow overlap can simply ignore the alternate identifier. Salvatore, I'm mostly speaking to you when I ask this but would welcome others' opinions too. Carl __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
Here is a compromise option. The pluggable IPAM will be optionally enabled in Kilo. We could introduce the restriction, but only when pluggable IPAM is enabled. Support for having a tenant with overlapping IP space, along with pluggable IPAM would wait until Liberty, when we can fully implement the address scope concept. This concept was discussed during the spec reviews of pluggable IPAM, and is simply adding a first-class object that represents a layer 3 address space. A subnet would belong to a specific scope, and any IP within a scope would be unique. To support the tenant with overlapping space, you would create multiple scopes for that tenant. This option maintains backward compatibility for existing deployments, while allowing us to improve the model moving forward. John On 3/11/15, 2:22 AM, Carl Baldwin c...@ecbaldwin.net wrote: On Tue, Mar 10, 2015 at 11:34 AM, Gabriel Bezerra gabri...@lsd.ufcg.edu.br wrote: Em 10.03.2015 14:24, Carl Baldwin escreveu: I'd vote for allowing against such restriction, but throwing an error in case of creating a router between the subnets. I can imagine a tenant running multiple instances of an application, each one with its own network that uses the same address range, to minimize configuration differences between them. I see your point but yuck! This isn't the place to skimp on configuration changes. Carl __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On 3/12/15, 12:46 AM, Carl Baldwin c...@ecbaldwin.net wrote: When talking with external IPAM to get a subnet, Neutron will pass both the cidr as the primary identifier and the subnet_id as an alternate identifier. External systems that do not allow overlap can Recall that IPAM driver instances are associated with a specific subnet pool. As long as we do not allow overlap within a *pool* this is not necessary. The pool will imply the scope (currently implicit, with one per tenant), which the driver/external system would use to differentiate the CIDR. As I mentioned in an earlier email, this would introduce the uniqueness constraint in Kilo, but only when pluggable IPAM is enabled. John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On 3/12/15, 2:33 AM, Carl Baldwin c...@ecbaldwin.net wrote: John, I think our proposals fit together nicely. This thread is about allowing overlap within a pool. I think it is fine for an external IPAM driver to disallow such overlap for now. However, the reference implementation must support it for backward compatibility and so my proposal will account for that. I was proposing that the reference driver not support it either, and we only handle that use case via the non-pluggable implementation in Kilo, waiting until Liberty to handle it in the pluggable implementation. However, I don't think it's particularly harmful to do it either way so I am OK with this. Thanks, John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On Wed, Mar 11, 2015 at 2:54 PM, John Belamaric jbelama...@infoblox.com wrote: I was proposing that the reference driver not support it either, and we only handle that use case via the non-pluggable implementation in Kilo, waiting until Liberty to handle it in the pluggable implementation. However, I don't think it's particularly harmful to do it either way so I am OK with this. I see. That is a bit different than what I had understood. After the responses on this thread and some other private responses, I'm not sure we'll ever be able to not support it for the reference driver. Carl __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
John, I think our proposals fit together nicely. This thread is about allowing overlap within a pool. I think it is fine for an external IPAM driver to disallow such overlap for now. However, the reference implementation must support it for backward compatibility and so my proposal will account for that. The consequence is that your driver will receive the subnet_id as a parameter to get_subnet. It is free to ignore this parameter. I'd still like to hear from Salvatore on this before I update the interface. Carl On Wed, Mar 11, 2015 at 11:08 AM, John Belamaric jbelama...@infoblox.com wrote: On 3/12/15, 12:46 AM, Carl Baldwin c...@ecbaldwin.net wrote: When talking with external IPAM to get a subnet, Neutron will pass both the cidr as the primary identifier and the subnet_id as an alternate identifier. External systems that do not allow overlap can Recall that IPAM driver instances are associated with a specific subnet pool. As long as we do not allow overlap within a *pool* this is not necessary. The pool will imply the scope (currently implicit, with one per tenant), which the driver/external system would use to differentiate the CIDR. As I mentioned in an earlier email, this would introduce the uniqueness constraint in Kilo, but only when pluggable IPAM is enabled. John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
My concern is that we are introducing new objects in Neutron that are scoped to a tenant and we don't have anything else like that right now. For example, I can create 100 3-tier topologies (router + 3 subnets/networks) with duplicated names, CIDRs, etc between all of them and it doesn't matter if I do it on 100 different tenants or all under the same tenant. Put a different way, tenants don't mean anything in the Neutron data model. They are just a tag to use to enforce quotas, policies, and filters on incoming API requests. Uniqueness shouldn't come from 'x' + tenant_id. It seems like what is being introduced here is going to fundamentally change that by forcing the creation of separate tenants to have overlapping IPs. I think the barrier should be very high to introduce anything that does that. Can you elaborate why tenants are used for uniqueness for IPAM instead of having a separate grouping mechanism? On Wed, Mar 11, 2015 at 3:24 PM, Ihar Hrachyshka ihrac...@redhat.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/10/2015 06:34 PM, Gabriel Bezerra wrote: Em 10.03.2015 14:24, Carl Baldwin escreveu: Neutron currently does not enforce the uniqueness, or non-overlap, of subnet cidrs within the address scope for a single tenant. For example, if a tenant chooses to use 10.0.0.0/24 on more than one subnet, he or she is free to do so. Problems will arise when trying to connect a router between these subnets but that is left up to the tenant to work out. In the current IPAM rework, we had decided to allow this overlap in the reference implementation for backward compatibility. However, we've hit a snag. It would be convenient to use the subnet cidr as the handle with which to refer to a previously allocated subnet when talking to IPAM. If overlap is allowed, this is not possible and we need to come up with another identifier such as Neutron's subnet_id or another unique IPAM specific ID. It could be a burden on an external IPAM system -- which does not allow overlap -- to work with a completely separate identifier for a subnet. I do not know of anyone using this capability (or mis-feature) of Neutron. I would hope that tenants are aware of the issues with trying to route between subnets with overlapping address spaces and would avoid it. Is this potential overlap something that we should really be worried about? Could we just add the assumption that subnets do not overlap within a tenant's scope? An important thing to note is that this topic is different than allowing overlap of cidrs between tenants. Neutron will continue to allow overlap of addresses between tenants and support the isolation of these address spaces. The IPAM rework will support this. Carl Baldwin I'd vote for allowing against such restriction, but throwing an error in case of creating a router between the subnets. I can imagine a tenant running multiple instances of an application, each one with its own network that uses the same address range, to minimize configuration differences between them. I agree with Gabriel on the matter. There is nothing inherently wrong about a tenant running multiple isolated network setups that use overlapping addresses (as there is nothing wrong about multiple tenants doing the same). There seems to be a value in disallowing overlapping subnets attached to the same router though. All in all, there seems to be no need to limit neutron API just because most external IPAM implementation do not seem to care about supporting the use case. It's also an issue that the change proposed is backwards incompatible. /Ihar -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJVAMCPAAoJEC5aWaUY1u57d/cH/A+nAuLrNaCPumjlOPJP90c3 4oSh0ioxEQw2uBRx1mWvcQle0M1U4Psy5CqIjeHgDhRlCNKB2gAYvm7/lCwZoCw7 pxLUerfZPFguNKYCUly1MYyo0gorycFgemoHKEFwHboDJfPdGYxdhW8HuemCClOG ZSeRzjO2rGaHU8XT+QWgI14UBiAu+XlQHy3UwUKEaffXOnja7noCU99/6d7+6TgF 5/RdFhpfn6pcRvLboiquo57w6N3q7moORgrGSMuP8cnMMTxo9/TLie+vMXmLPobB YmeG1ar1q0ouGKb6ZG7KokUyl6CdpgowZ6bqetPELjBLIO+uhcsJ6g9NvRl4T9o= =WQ3Q -END PGP SIGNATURE- __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
This has been settled and we're not moving forward with it for Kilo. I agree tenants are an administrative concept, not a networking one so using them for uniqueness doesn't really make sense. In Liberty we are proposing a new grouping mechanism, as you call it, specifically for the purpose of defining uniqueness - address scopes. This would be owned by a tenant but could be shared across tenants. It's still in the early stages of definition though, and more discussion is needed but should probably wait until after Kilo is out! From: Kevin Benton blak...@gmail.commailto:blak...@gmail.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Thursday, March 12, 2015 at 6:38 AM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant My concern is that we are introducing new objects in Neutron that are scoped to a tenant and we don't have anything else like that right now. For example, I can create 100 3-tier topologies (router + 3 subnets/networks) with duplicated names, CIDRs, etc between all of them and it doesn't matter if I do it on 100 different tenants or all under the same tenant. Put a different way, tenants don't mean anything in the Neutron data model. They are just a tag to use to enforce quotas, policies, and filters on incoming API requests. Uniqueness shouldn't come from 'x' + tenant_id. It seems like what is being introduced here is going to fundamentally change that by forcing the creation of separate tenants to have overlapping IPs. I think the barrier should be very high to introduce anything that does that. Can you elaborate why tenants are used for uniqueness for IPAM instead of having a separate grouping mechanism? On Wed, Mar 11, 2015 at 3:24 PM, Ihar Hrachyshka ihrac...@redhat.commailto:ihrac...@redhat.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/10/2015 06:34 PM, Gabriel Bezerra wrote: Em 10.03.2015 14:24, Carl Baldwin escreveu: Neutron currently does not enforce the uniqueness, or non-overlap, of subnet cidrs within the address scope for a single tenant. For example, if a tenant chooses to use 10.0.0.0/24http://10.0.0.0/24 on more than one subnet, he or she is free to do so. Problems will arise when trying to connect a router between these subnets but that is left up to the tenant to work out. In the current IPAM rework, we had decided to allow this overlap in the reference implementation for backward compatibility. However, we've hit a snag. It would be convenient to use the subnet cidr as the handle with which to refer to a previously allocated subnet when talking to IPAM. If overlap is allowed, this is not possible and we need to come up with another identifier such as Neutron's subnet_id or another unique IPAM specific ID. It could be a burden on an external IPAM system -- which does not allow overlap -- to work with a completely separate identifier for a subnet. I do not know of anyone using this capability (or mis-feature) of Neutron. I would hope that tenants are aware of the issues with trying to route between subnets with overlapping address spaces and would avoid it. Is this potential overlap something that we should really be worried about? Could we just add the assumption that subnets do not overlap within a tenant's scope? An important thing to note is that this topic is different than allowing overlap of cidrs between tenants. Neutron will continue to allow overlap of addresses between tenants and support the isolation of these address spaces. The IPAM rework will support this. Carl Baldwin I'd vote for allowing against such restriction, but throwing an error in case of creating a router between the subnets. I can imagine a tenant running multiple instances of an application, each one with its own network that uses the same address range, to minimize configuration differences between them. I agree with Gabriel on the matter. There is nothing inherently wrong about a tenant running multiple isolated network setups that use overlapping addresses (as there is nothing wrong about multiple tenants doing the same). There seems to be a value in disallowing overlapping subnets attached to the same router though. All in all, there seems to be no need to limit neutron API just because most external IPAM implementation do not seem to care about supporting the use case. It's also an issue that the change proposed is backwards incompatible. /Ihar -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJVAMCPAAoJEC5aWaUY1u57d/cH/A+nAuLrNaCPumjlOPJP90c3 4oSh0ioxEQw2uBRx1mWvcQle0M1U4Psy5CqIjeHgDhRlCNKB2gAYvm7/lCwZoCw7 pxLUerfZPFguNKYCUly1MYyo0gorycFgemoHKEFwHboDJfPdGYxdhW8HuemCClOG ZSeRzjO2rGaHU8XT+QWgI14UBiAu
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/10/2015 06:34 PM, Gabriel Bezerra wrote: Em 10.03.2015 14:24, Carl Baldwin escreveu: Neutron currently does not enforce the uniqueness, or non-overlap, of subnet cidrs within the address scope for a single tenant. For example, if a tenant chooses to use 10.0.0.0/24 on more than one subnet, he or she is free to do so. Problems will arise when trying to connect a router between these subnets but that is left up to the tenant to work out. In the current IPAM rework, we had decided to allow this overlap in the reference implementation for backward compatibility. However, we've hit a snag. It would be convenient to use the subnet cidr as the handle with which to refer to a previously allocated subnet when talking to IPAM. If overlap is allowed, this is not possible and we need to come up with another identifier such as Neutron's subnet_id or another unique IPAM specific ID. It could be a burden on an external IPAM system -- which does not allow overlap -- to work with a completely separate identifier for a subnet. I do not know of anyone using this capability (or mis-feature) of Neutron. I would hope that tenants are aware of the issues with trying to route between subnets with overlapping address spaces and would avoid it. Is this potential overlap something that we should really be worried about? Could we just add the assumption that subnets do not overlap within a tenant's scope? An important thing to note is that this topic is different than allowing overlap of cidrs between tenants. Neutron will continue to allow overlap of addresses between tenants and support the isolation of these address spaces. The IPAM rework will support this. Carl Baldwin I'd vote for allowing against such restriction, but throwing an error in case of creating a router between the subnets. I can imagine a tenant running multiple instances of an application, each one with its own network that uses the same address range, to minimize configuration differences between them. I agree with Gabriel on the matter. There is nothing inherently wrong about a tenant running multiple isolated network setups that use overlapping addresses (as there is nothing wrong about multiple tenants doing the same). There seems to be a value in disallowing overlapping subnets attached to the same router though. All in all, there seems to be no need to limit neutron API just because most external IPAM implementation do not seem to care about supporting the use case. It's also an issue that the change proposed is backwards incompatible. /Ihar -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJVAMCPAAoJEC5aWaUY1u57d/cH/A+nAuLrNaCPumjlOPJP90c3 4oSh0ioxEQw2uBRx1mWvcQle0M1U4Psy5CqIjeHgDhRlCNKB2gAYvm7/lCwZoCw7 pxLUerfZPFguNKYCUly1MYyo0gorycFgemoHKEFwHboDJfPdGYxdhW8HuemCClOG ZSeRzjO2rGaHU8XT+QWgI14UBiAu+XlQHy3UwUKEaffXOnja7noCU99/6d7+6TgF 5/RdFhpfn6pcRvLboiquo57w6N3q7moORgrGSMuP8cnMMTxo9/TLie+vMXmLPobB YmeG1ar1q0ouGKb6ZG7KokUyl6CdpgowZ6bqetPELjBLIO+uhcsJ6g9NvRl4T9o= =WQ3Q -END PGP SIGNATURE- __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On Tue, Mar 10, 2015 at 10:38 PM, Gabriel Bezerra gabri...@lsd.ufcg.edu.br wrote: Em 10.03.2015 14:34, Gabriel Bezerra escreveu: Em 10.03.2015 14:24, Carl Baldwin escreveu: Neutron currently does not enforce the uniqueness, or non-overlap, of subnet cidrs within the address scope for a single tenant. For example, if a tenant chooses to use 10.0.0.0/24 on more than one subnet, he or she is free to do so. Problems will arise when trying to connect a router between these subnets but that is left up to the tenant to work out. In the current IPAM rework, we had decided to allow this overlap in the reference implementation for backward compatibility. However, we've hit a snag. It would be convenient to use the subnet cidr as the handle with which to refer to a previously allocated subnet when talking to IPAM. If overlap is allowed, this is not possible and we need to come up with another identifier such as Neutron's subnet_id or another unique IPAM specific ID. It could be a burden on an external IPAM system -- which does not allow overlap -- to work with a completely separate identifier for a subnet. I do not know of anyone using this capability (or mis-feature) of Neutron. I would hope that tenants are aware of the issues with trying to route between subnets with overlapping address spaces and would avoid it. Is this potential overlap something that we should really be worried about? Could we just add the assumption that subnets do not overlap within a tenant's scope? An important thing to note is that this topic is different than allowing overlap of cidrs between tenants. Neutron will continue to allow overlap of addresses between tenants and support the isolation of these address spaces. The IPAM rework will support this. Carl Baldwin I'd vote for allowing against such restriction, but throwing an error in case of creating a router between the subnets. Fixing my previous e-mail: I'd vote against such restriction, but throwing an error in case of creating a router between the subnets that overlap. +1 to Gabriel's suggestion. Multiple routers and multiple subnets with overlapping IPs is a perfectly valid scenario and is used in some blueprints; for instance PLUMgrid plugin supports it. Throwing an error for overlapping IPs on Router interfaces seems like the right approach. I can imagine a tenant running multiple instances of an application, each one with its own network that uses the same address range, to minimize configuration differences between them. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
Gabriel Bezerra gabri...@lsd.ufcg.edu.br wrote on 03/10/2015 12:34:30 PM: Em 10.03.2015 14:24, Carl Baldwin escreveu: Neutron currently does not enforce the uniqueness, or non-overlap, of subnet cidrs within the address scope for a single tenant. For example, if a tenant chooses to use 10.0.0.0/24 on more than one subnet, he or she is free to do so. Problems will arise when trying to connect a router between these subnets but that is left up to the tenant to work out. In the current IPAM rework, we had decided to allow this overlap in the reference implementation for backward compatibility. However, we've hit a snag. It would be convenient to use the subnet cidr as the handle with which to refer to a previously allocated subnet when talking to IPAM. If overlap is allowed, this is not possible and we need to come up with another identifier such as Neutron's subnet_id or another unique IPAM specific ID. It could be a burden on an external IPAM system -- which does not allow overlap -- to work with a completely separate identifier for a subnet. I do not know of anyone using this capability (or mis-feature) of Neutron. I would hope that tenants are aware of the issues with trying to route between subnets with overlapping address spaces and would avoid it. Is this potential overlap something that we should really be worried about? Could we just add the assumption that subnets do not overlap within a tenant's scope? An important thing to note is that this topic is different than allowing overlap of cidrs between tenants. Neutron will continue to allow overlap of addresses between tenants and support the isolation of these address spaces. The IPAM rework will support this. Carl Baldwin I'd vote for allowing against such restriction, but throwing an error in case of creating a router between the subnets. I can imagine a tenant running multiple instances of an application, each one with its own network that uses the same address range, to minimize configuration differences between them. While I'd personally like to see this be restricted (Carl's position), I know of at least one existence proof where management applications are doing precisely what Gabriel is suggesting - reusing the same address range to minimize the configuration differences. Ryan Moats__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
Em 10.03.2015 14:34, Gabriel Bezerra escreveu: Em 10.03.2015 14:24, Carl Baldwin escreveu: Neutron currently does not enforce the uniqueness, or non-overlap, of subnet cidrs within the address scope for a single tenant. For example, if a tenant chooses to use 10.0.0.0/24 on more than one subnet, he or she is free to do so. Problems will arise when trying to connect a router between these subnets but that is left up to the tenant to work out. In the current IPAM rework, we had decided to allow this overlap in the reference implementation for backward compatibility. However, we've hit a snag. It would be convenient to use the subnet cidr as the handle with which to refer to a previously allocated subnet when talking to IPAM. If overlap is allowed, this is not possible and we need to come up with another identifier such as Neutron's subnet_id or another unique IPAM specific ID. It could be a burden on an external IPAM system -- which does not allow overlap -- to work with a completely separate identifier for a subnet. I do not know of anyone using this capability (or mis-feature) of Neutron. I would hope that tenants are aware of the issues with trying to route between subnets with overlapping address spaces and would avoid it. Is this potential overlap something that we should really be worried about? Could we just add the assumption that subnets do not overlap within a tenant's scope? An important thing to note is that this topic is different than allowing overlap of cidrs between tenants. Neutron will continue to allow overlap of addresses between tenants and support the isolation of these address spaces. The IPAM rework will support this. Carl Baldwin I'd vote for allowing against such restriction, but throwing an error in case of creating a router between the subnets. Fixing my previous e-mail: I'd vote against such restriction, but throwing an error in case of creating a router between the subnets that overlap. I can imagine a tenant running multiple instances of an application, each one with its own network that uses the same address range, to minimize configuration differences between them. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
Em 10.03.2015 14:24, Carl Baldwin escreveu: Neutron currently does not enforce the uniqueness, or non-overlap, of subnet cidrs within the address scope for a single tenant. For example, if a tenant chooses to use 10.0.0.0/24 on more than one subnet, he or she is free to do so. Problems will arise when trying to connect a router between these subnets but that is left up to the tenant to work out. In the current IPAM rework, we had decided to allow this overlap in the reference implementation for backward compatibility. However, we've hit a snag. It would be convenient to use the subnet cidr as the handle with which to refer to a previously allocated subnet when talking to IPAM. If overlap is allowed, this is not possible and we need to come up with another identifier such as Neutron's subnet_id or another unique IPAM specific ID. It could be a burden on an external IPAM system -- which does not allow overlap -- to work with a completely separate identifier for a subnet. I do not know of anyone using this capability (or mis-feature) of Neutron. I would hope that tenants are aware of the issues with trying to route between subnets with overlapping address spaces and would avoid it. Is this potential overlap something that we should really be worried about? Could we just add the assumption that subnets do not overlap within a tenant's scope? An important thing to note is that this topic is different than allowing overlap of cidrs between tenants. Neutron will continue to allow overlap of addresses between tenants and support the isolation of these address spaces. The IPAM rework will support this. Carl Baldwin I'd vote for allowing against such restriction, but throwing an error in case of creating a router between the subnets. I can imagine a tenant running multiple instances of an application, each one with its own network that uses the same address range, to minimize configuration differences between them. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][IPAM] Uniqueness of subnets within a tenant
On Tue, Mar 10, 2015 at 11:34 AM, Gabriel Bezerra gabri...@lsd.ufcg.edu.br wrote: Em 10.03.2015 14:24, Carl Baldwin escreveu: I'd vote for allowing against such restriction, but throwing an error in case of creating a router between the subnets. I can imagine a tenant running multiple instances of an application, each one with its own network that uses the same address range, to minimize configuration differences between them. I see your point but yuck! This isn't the place to skimp on configuration changes. Carl __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
