Re: [openstack-dev] [Nova][TripleO] Nested resources
Hi Kevin, 2013/12/2 Fox, Kevin M kevin@pnnl.gov Hi all, I just want to run a crazy idea up the flag pole. TripleO has the concept of an under and over cloud. In starting to experiment with Docker, I see a pattern start to emerge. * As a User, I may want to allocate a BareMetal node so that it is entirely mine. I may want to run multiple VM's on it to reduce my own cost. Now I have to manage the BareMetal nodes myself or nest OpenStack into them. * As a User, I may want to allocate a VM. I then want to run multiple Docker containers on it to use it more efficiently. Now I have to manage the VM's myself or nest OpenStack into them. * As a User, I may want to allocate a BareMetal node so that it is entirely mine. I then want to run multiple Docker containers on it to use it more efficiently. Now I have to manage the BareMetal nodes myself or nest OpenStack into them. I think this can then be generalized to: As a User, I would like to ask for resources of one type (One AZ?), and be able to delegate resources back to Nova so that I can use Nova to subdivide and give me access to my resources as a different type. (As a different AZ?) Have you ever heard about Climate [1], our Stackforge project for reservations as a service in Openstack ? I'm just saying that because your usecases seem pretty identical as the ones we defined for starting the project... Climate is aiming to reserve both virtual (instances, stacks, etc.) and physical (compute hosts) resources [2]. There is no overhead with hypervisors, that's only matter of saying 'this node is for me, I'll be booting anything I want on it, provided the hypervisor would be KVM, Xen-API, or Baremetal (of course, with the Baremetal driver, you would then only get a lock) That said, I'm just talking about Climate for making sure we don't generate silos and duplicate stuff. I still have to think about how we can plug TripleO resources as native resources for Climate, so in that case TripleO would just have to call Climate for provisioning the resources. -Sylvain [1] : https://launchpad.net/climate [2] : https://wiki.openstack.org/wiki/Blueprint-nova-planned-resource-reservation-api I think this could potentially cover some of the TripleO stuff without needing an over/under cloud. For that use case, all the BareMetal nodes could be added to Nova as such, allocated by the services tenant as running a nested VM image type resource stack, and then made available to all tenants. Sys admins then could dynamically shift resources from VM providing nodes to BareMetal Nodes and back as needed. This allows a user to allocate some raw resources as a group, then schedule higher level services to run only in that group, all with the existing api. Just how crazy an idea is this? Thanks, Kevin ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][TripleO] Nested resources
Excerpts from Sylvain Bauza's message of 2013-12-14 06:23:48 -0800: 2013/12/9 Clint Byrum cl...@fewbar.com Excerpts from Fox, Kevin M's message of 2013-12-09 09:34:06 -0800: I'm thinking more generic: The cloud provider will provide one or more suballocating images. The one Triple O uses to take a bare metal node and make vm's available would be the obvious one to make available initially. I think that one should not have a security concern since it is already being used in that way safely. I like where you're going with this, in that the cloud should eventually become self aware enough to be able to privision the baremetal resources it has and spin nova up on them. I do think that is quite far out. Right now, we have two nova's.. an undercloud nova which owns all the baremetal, and an overcloud nova which owns all the vms. This is definitely nested, but there is a hard line between the two. For many people, that hard line is a feature. For others, it is a bug. :) Could we imagine that an end-user would like to provision one undercloud host plus a certain number of overcloud nodes so that the Scheduler for undercloud Nova would deny other hosts but the ones provisioned ? Yes I could imagine that. I also imagine that does not require any special knowledge of the undercloud that the overcloud's nova API doesn't already have access to. The host is a thing in the overcloud after all. As a contrary, Scheduler for other undercloud Nova's need to deny the provisioning of the nodes hosted by another tenant than the requester... I played with TripleO a few months ago (August/September, before the merge with Tuskar) so that's a bit unclear for me, but I'm just saying we could potentially achieve this by using Climate which does deploy its own Scheduler Filter for making sure the proper hosts are booted. Currently undercloud nova has one tenant: the overcloud operator. It is single tenant, which means it has less complexity, but that also means you can't hand hardware out directly to multiple tenants. That is why I say that some consider it a feature, and some a bug. IMO that is how it should remain, and we should just enhance systems like Climate to be more aware of the topology of the hosts which are already an entity in the overcloud. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][TripleO] Nested resources
On Dec 5, 2013, at 8:11 PM, Fox, Kevin M wrote: I think the security issue can be handled by not actually giving the underlying resource to the user in the first place. So, for example, if I wanted a bare metal node's worth of resource for my own containering, I'd ask for a bare metal node and use a blessed image that contains docker+nova bits that would hook back to the cloud. I wouldn't be able to login to it, but containers started on it would be able to access my tenant's networks. All access to it would have to be through nova suballocations. The bare resource would count against my quotas, but nothing run under it. So this would be an extremely light weight hypervisor alternative, then? It's interesting because bare-metal-to-tenant security issues are tricky to overcome. -k Come to think of it, this sounds somewhat similar to what is planned for Neutron service vm's. They count against the user's quota on one level but not all access is directly given to the user. Maybe some of the same implementation bits could be used. Thanks, Kevin From: Mark McLoughlin [mar...@redhat.com] Sent: Thursday, December 05, 2013 1:53 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova][TripleO] Nested resources Hi Kevin, On Mon, 2013-12-02 at 12:39 -0800, Fox, Kevin M wrote: Hi all, I just want to run a crazy idea up the flag pole. TripleO has the concept of an under and over cloud. In starting to experiment with Docker, I see a pattern start to emerge. * As a User, I may want to allocate a BareMetal node so that it is entirely mine. I may want to run multiple VM's on it to reduce my own cost. Now I have to manage the BareMetal nodes myself or nest OpenStack into them. * As a User, I may want to allocate a VM. I then want to run multiple Docker containers on it to use it more efficiently. Now I have to manage the VM's myself or nest OpenStack into them. * As a User, I may want to allocate a BareMetal node so that it is entirely mine. I then want to run multiple Docker containers on it to use it more efficiently. Now I have to manage the BareMetal nodes myself or nest OpenStack into them. I think this can then be generalized to: As a User, I would like to ask for resources of one type (One AZ?), and be able to delegate resources back to Nova so that I can use Nova to subdivide and give me access to my resources as a different type. (As a different AZ?) I think this could potentially cover some of the TripleO stuff without needing an over/under cloud. For that use case, all the BareMetal nodes could be added to Nova as such, allocated by the services tenant as running a nested VM image type resource stack, and then made available to all tenants. Sys admins then could dynamically shift resources from VM providing nodes to BareMetal Nodes and back as needed. This allows a user to allocate some raw resources as a group, then schedule higher level services to run only in that group, all with the existing api. Just how crazy an idea is this? FWIW, I don't think it's a crazy idea at all - indeed I mumbled something similar a few times in conversation with random people over the past few months :) With the increasing interest in containers, it makes a tonne of sense - you provision a number of VMs and now you want to carve them up by allocating containers on them. Right now, you'd need to run your own instance of Nova for that ... which is far too heavyweight. It is a little crazy in the sense that it's a tonne of work, though. There's not a whole lot of point in discussing it too much until someone shows signs of wanting to implement it :) One problem is how the API would model this nesting, another problem is making the scheduler aware that some nodes are only available to the tenant which owns them but maybe a bigger problem is the security model around allowing a node managed by an untrusted become a compute node. Mark. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][TripleO] Nested resources
I'm thinking more generic: The cloud provider will provide one or more suballocating images. The one Triple O uses to take a bare metal node and make vm's available would be the obvious one to make available initially. I think that one should not have a security concern since it is already being used in that way safely. I think a docker based one shouldn't have the safety concern either, since I think docker containerizes network resources too? I could be wrong though. Thanks, Kevin From: Keith Basil [kba...@redhat.com] Sent: Monday, December 09, 2013 8:50 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova][TripleO] Nested resources On Dec 5, 2013, at 8:11 PM, Fox, Kevin M wrote: I think the security issue can be handled by not actually giving the underlying resource to the user in the first place. So, for example, if I wanted a bare metal node's worth of resource for my own containering, I'd ask for a bare metal node and use a blessed image that contains docker+nova bits that would hook back to the cloud. I wouldn't be able to login to it, but containers started on it would be able to access my tenant's networks. All access to it would have to be through nova suballocations. The bare resource would count against my quotas, but nothing run under it. So this would be an extremely light weight hypervisor alternative, then? It's interesting because bare-metal-to-tenant security issues are tricky to overcome. -k Come to think of it, this sounds somewhat similar to what is planned for Neutron service vm's. They count against the user's quota on one level but not all access is directly given to the user. Maybe some of the same implementation bits could be used. Thanks, Kevin From: Mark McLoughlin [mar...@redhat.com] Sent: Thursday, December 05, 2013 1:53 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova][TripleO] Nested resources Hi Kevin, On Mon, 2013-12-02 at 12:39 -0800, Fox, Kevin M wrote: Hi all, I just want to run a crazy idea up the flag pole. TripleO has the concept of an under and over cloud. In starting to experiment with Docker, I see a pattern start to emerge. * As a User, I may want to allocate a BareMetal node so that it is entirely mine. I may want to run multiple VM's on it to reduce my own cost. Now I have to manage the BareMetal nodes myself or nest OpenStack into them. * As a User, I may want to allocate a VM. I then want to run multiple Docker containers on it to use it more efficiently. Now I have to manage the VM's myself or nest OpenStack into them. * As a User, I may want to allocate a BareMetal node so that it is entirely mine. I then want to run multiple Docker containers on it to use it more efficiently. Now I have to manage the BareMetal nodes myself or nest OpenStack into them. I think this can then be generalized to: As a User, I would like to ask for resources of one type (One AZ?), and be able to delegate resources back to Nova so that I can use Nova to subdivide and give me access to my resources as a different type. (As a different AZ?) I think this could potentially cover some of the TripleO stuff without needing an over/under cloud. For that use case, all the BareMetal nodes could be added to Nova as such, allocated by the services tenant as running a nested VM image type resource stack, and then made available to all tenants. Sys admins then could dynamically shift resources from VM providing nodes to BareMetal Nodes and back as needed. This allows a user to allocate some raw resources as a group, then schedule higher level services to run only in that group, all with the existing api. Just how crazy an idea is this? FWIW, I don't think it's a crazy idea at all - indeed I mumbled something similar a few times in conversation with random people over the past few months :) With the increasing interest in containers, it makes a tonne of sense - you provision a number of VMs and now you want to carve them up by allocating containers on them. Right now, you'd need to run your own instance of Nova for that ... which is far too heavyweight. It is a little crazy in the sense that it's a tonne of work, though. There's not a whole lot of point in discussing it too much until someone shows signs of wanting to implement it :) One problem is how the API would model this nesting, another problem is making the scheduler aware that some nodes are only available to the tenant which owns them but maybe a bigger problem is the security model around allowing a node managed by an untrusted become a compute node. Mark. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman
Re: [openstack-dev] [Nova][TripleO] Nested resources
Excerpts from Fox, Kevin M's message of 2013-12-09 09:34:06 -0800: I'm thinking more generic: The cloud provider will provide one or more suballocating images. The one Triple O uses to take a bare metal node and make vm's available would be the obvious one to make available initially. I think that one should not have a security concern since it is already being used in that way safely. I like where you're going with this, in that the cloud should eventually become self aware enough to be able to privision the baremetal resources it has and spin nova up on them. I do think that is quite far out. Right now, we have two nova's.. an undercloud nova which owns all the baremetal, and an overcloud nova which owns all the vms. This is definitely nested, but there is a hard line between the two. For many people, that hard line is a feature. For others, it is a bug. :) I think a docker based one shouldn't have the safety concern either, since I think docker containerizes network resources too? I could be wrong though. The baremetal-to-tenant issues have little to do with networking. They are firmware problems. Root just has too much power on baremetal. Somebody should make some hardware which defends against that. For now the best thing is virtualization extensions. Docker isn't really going to fix that. The containerization that is available is good, but does not do nearly as much as true virtualization does to isolate the user from the hardware. There's still a single kernel there, and thus, if you can trick that kernel, you can own the whole box. I've heard it descried as a little better than chroot. AFAIK, the people using containers for multi-tenant are doing so by leveraging kernel security modules heavily. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][TripleO] Nested resources
On 6 December 2013 14:11, Fox, Kevin M kevin@pnnl.gov wrote: I think the security issue can be handled by not actually giving the underlying resource to the user in the first place. So, for example, if I wanted a bare metal node's worth of resource for my own containering, I'd ask for a bare metal node and use a blessed image that contains docker+nova bits that would hook back to the cloud. I wouldn't be able to login to it, but containers started on it would be able to access my tenant's networks. All access to it would have to be through nova suballocations. The bare resource would count against my quotas, but nothing run under it. Come to think of it, this sounds somewhat similar to what is planned for Neutron service vm's. They count against the user's quota on one level but not all access is directly given to the user. Maybe some of the same implementation bits could be used. This is a super interesting discussion - thanks for kicking it off. I think it would be fantastic to be able to use containers for deploying the cloud rather than full images while still running entirely OpenStack control up and down the stack. Briefly, what we need to be able to do that is: - the ability to bring up an all in one node with everything on it to 'seed' the environment. - we currently do that by building a disk image, and manually running virsh to start it - the ability to reboot a machine *with no other machines running* - we need to be able to power off and on a datacentre - and have the containers on it come up correctly configured, networking working, running etc. - we explicitly want to be just using OpenStack APIs for all the deployment operations after the seed is up; so no direct use of lxc or docker or whathaveyou. Cheers, Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][TripleO] Nested resources
On Tue, 2013-12-10 at 09:40 +1300, Robert Collins wrote: On 6 December 2013 14:11, Fox, Kevin M kevin@pnnl.gov wrote: I think the security issue can be handled by not actually giving the underlying resource to the user in the first place. So, for example, if I wanted a bare metal node's worth of resource for my own containering, I'd ask for a bare metal node and use a blessed image that contains docker+nova bits that would hook back to the cloud. I wouldn't be able to login to it, but containers started on it would be able to access my tenant's networks. All access to it would have to be through nova suballocations. The bare resource would count against my quotas, but nothing run under it. Come to think of it, this sounds somewhat similar to what is planned for Neutron service vm's. They count against the user's quota on one level but not all access is directly given to the user. Maybe some of the same implementation bits could be used. This is a super interesting discussion - thanks for kicking it off. I think it would be fantastic to be able to use containers for deploying the cloud rather than full images while still running entirely OpenStack control up and down the stack. Where I think it gets really interesting is to be able to auto-scale controller services (think nova-api based on request latency) in small increments just you'd expect to be able to manage a scale-out app on a cloud. i.e. our overcloud Heat stack would allocate some baremetal machines, but then just schedule the controller services to run in small containers (or VMs) on any of those machines, and then have them auto-scale. Briefly, what we need to be able to do that is: - the ability to bring up an all in one node with everything on it to 'seed' the environment. - we currently do that by building a disk image, and manually running virsh to start it I'm not sure that would need to change. - the ability to reboot a machine *with no other machines running* - we need to be able to power off and on a datacentre - and have the containers on it come up correctly configured, networking working, running etc. That's tricky because your undercloud Nova DB/conductor needs to be available for the machine to know what services it's supposed to be running. It sounds like a reasonable thing to want even for standard KVM compute nodes too, though. - we explicitly want to be just using OpenStack APIs for all the deployment operations after the seed is up; so no direct use of lxc or docker or whathaveyou. Yes. Mark. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][TripleO] Nested resources
Along the same lines and while we're talking crazy ideas, one use case where a user might want to allocate entire nodes would be if TripleO were used to manage an ARM rack. The use cases aren't identical between cloud and ARM, but they are similar. So for a rack of 1000 nodes, there is benefit in certain cases for a user not only taking an entire node, but a collection of nodes co-located in the same rack to take advantage of the rack fabric. Again, crazy ideas and probably outside of the scope of things we want to bite off immediately. But as we're in the early stages of the Tuskar data and security models, it might make sense to at least keep in mind how we could play in this area as well. On 12/05/2013 08:11 PM, Fox, Kevin M wrote: I think the security issue can be handled by not actually giving the underlying resource to the user in the first place. So, for example, if I wanted a bare metal node's worth of resource for my own containering, I'd ask for a bare metal node and use a blessed image that contains docker+nova bits that would hook back to the cloud. I wouldn't be able to login to it, but containers started on it would be able to access my tenant's networks. All access to it would have to be through nova suballocations. The bare resource would count against my quotas, but nothing run under it. Come to think of it, this sounds somewhat similar to what is planned for Neutron service vm's. They count against the user's quota on one level but not all access is directly given to the user. Maybe some of the same implementation bits could be used. Thanks, Kevin From: Mark McLoughlin [mar...@redhat.com] Sent: Thursday, December 05, 2013 1:53 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova][TripleO] Nested resources Hi Kevin, On Mon, 2013-12-02 at 12:39 -0800, Fox, Kevin M wrote: Hi all, I just want to run a crazy idea up the flag pole. TripleO has the concept of an under and over cloud. In starting to experiment with Docker, I see a pattern start to emerge. * As a User, I may want to allocate a BareMetal node so that it is entirely mine. I may want to run multiple VM's on it to reduce my own cost. Now I have to manage the BareMetal nodes myself or nest OpenStack into them. * As a User, I may want to allocate a VM. I then want to run multiple Docker containers on it to use it more efficiently. Now I have to manage the VM's myself or nest OpenStack into them. * As a User, I may want to allocate a BareMetal node so that it is entirely mine. I then want to run multiple Docker containers on it to use it more efficiently. Now I have to manage the BareMetal nodes myself or nest OpenStack into them. I think this can then be generalized to: As a User, I would like to ask for resources of one type (One AZ?), and be able to delegate resources back to Nova so that I can use Nova to subdivide and give me access to my resources as a different type. (As a different AZ?) I think this could potentially cover some of the TripleO stuff without needing an over/under cloud. For that use case, all the BareMetal nodes could be added to Nova as such, allocated by the services tenant as running a nested VM image type resource stack, and then made available to all tenants. Sys admins then could dynamically shift resources from VM providing nodes to BareMetal Nodes and back as needed. This allows a user to allocate some raw resources as a group, then schedule higher level services to run only in that group, all with the existing api. Just how crazy an idea is this? FWIW, I don't think it's a crazy idea at all - indeed I mumbled something similar a few times in conversation with random people over the past few months :) With the increasing interest in containers, it makes a tonne of sense - you provision a number of VMs and now you want to carve them up by allocating containers on them. Right now, you'd need to run your own instance of Nova for that ... which is far too heavyweight. It is a little crazy in the sense that it's a tonne of work, though. There's not a whole lot of point in discussing it too much until someone shows signs of wanting to implement it :) One problem is how the API would model this nesting, another problem is making the scheduler aware that some nodes are only available to the tenant which owns them but maybe a bigger problem is the security model around allowing a node managed by an untrusted become a compute node. Mark. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev
Re: [openstack-dev] [Nova][TripleO] Nested resources
Hi Kevin, On Mon, 2013-12-02 at 12:39 -0800, Fox, Kevin M wrote: Hi all, I just want to run a crazy idea up the flag pole. TripleO has the concept of an under and over cloud. In starting to experiment with Docker, I see a pattern start to emerge. * As a User, I may want to allocate a BareMetal node so that it is entirely mine. I may want to run multiple VM's on it to reduce my own cost. Now I have to manage the BareMetal nodes myself or nest OpenStack into them. * As a User, I may want to allocate a VM. I then want to run multiple Docker containers on it to use it more efficiently. Now I have to manage the VM's myself or nest OpenStack into them. * As a User, I may want to allocate a BareMetal node so that it is entirely mine. I then want to run multiple Docker containers on it to use it more efficiently. Now I have to manage the BareMetal nodes myself or nest OpenStack into them. I think this can then be generalized to: As a User, I would like to ask for resources of one type (One AZ?), and be able to delegate resources back to Nova so that I can use Nova to subdivide and give me access to my resources as a different type. (As a different AZ?) I think this could potentially cover some of the TripleO stuff without needing an over/under cloud. For that use case, all the BareMetal nodes could be added to Nova as such, allocated by the services tenant as running a nested VM image type resource stack, and then made available to all tenants. Sys admins then could dynamically shift resources from VM providing nodes to BareMetal Nodes and back as needed. This allows a user to allocate some raw resources as a group, then schedule higher level services to run only in that group, all with the existing api. Just how crazy an idea is this? FWIW, I don't think it's a crazy idea at all - indeed I mumbled something similar a few times in conversation with random people over the past few months :) With the increasing interest in containers, it makes a tonne of sense - you provision a number of VMs and now you want to carve them up by allocating containers on them. Right now, you'd need to run your own instance of Nova for that ... which is far too heavyweight. It is a little crazy in the sense that it's a tonne of work, though. There's not a whole lot of point in discussing it too much until someone shows signs of wanting to implement it :) One problem is how the API would model this nesting, another problem is making the scheduler aware that some nodes are only available to the tenant which owns them but maybe a bigger problem is the security model around allowing a node managed by an untrusted become a compute node. Mark. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][TripleO] Nested resources
I think the security issue can be handled by not actually giving the underlying resource to the user in the first place. So, for example, if I wanted a bare metal node's worth of resource for my own containering, I'd ask for a bare metal node and use a blessed image that contains docker+nova bits that would hook back to the cloud. I wouldn't be able to login to it, but containers started on it would be able to access my tenant's networks. All access to it would have to be through nova suballocations. The bare resource would count against my quotas, but nothing run under it. Come to think of it, this sounds somewhat similar to what is planned for Neutron service vm's. They count against the user's quota on one level but not all access is directly given to the user. Maybe some of the same implementation bits could be used. Thanks, Kevin From: Mark McLoughlin [mar...@redhat.com] Sent: Thursday, December 05, 2013 1:53 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova][TripleO] Nested resources Hi Kevin, On Mon, 2013-12-02 at 12:39 -0800, Fox, Kevin M wrote: Hi all, I just want to run a crazy idea up the flag pole. TripleO has the concept of an under and over cloud. In starting to experiment with Docker, I see a pattern start to emerge. * As a User, I may want to allocate a BareMetal node so that it is entirely mine. I may want to run multiple VM's on it to reduce my own cost. Now I have to manage the BareMetal nodes myself or nest OpenStack into them. * As a User, I may want to allocate a VM. I then want to run multiple Docker containers on it to use it more efficiently. Now I have to manage the VM's myself or nest OpenStack into them. * As a User, I may want to allocate a BareMetal node so that it is entirely mine. I then want to run multiple Docker containers on it to use it more efficiently. Now I have to manage the BareMetal nodes myself or nest OpenStack into them. I think this can then be generalized to: As a User, I would like to ask for resources of one type (One AZ?), and be able to delegate resources back to Nova so that I can use Nova to subdivide and give me access to my resources as a different type. (As a different AZ?) I think this could potentially cover some of the TripleO stuff without needing an over/under cloud. For that use case, all the BareMetal nodes could be added to Nova as such, allocated by the services tenant as running a nested VM image type resource stack, and then made available to all tenants. Sys admins then could dynamically shift resources from VM providing nodes to BareMetal Nodes and back as needed. This allows a user to allocate some raw resources as a group, then schedule higher level services to run only in that group, all with the existing api. Just how crazy an idea is this? FWIW, I don't think it's a crazy idea at all - indeed I mumbled something similar a few times in conversation with random people over the past few months :) With the increasing interest in containers, it makes a tonne of sense - you provision a number of VMs and now you want to carve them up by allocating containers on them. Right now, you'd need to run your own instance of Nova for that ... which is far too heavyweight. It is a little crazy in the sense that it's a tonne of work, though. There's not a whole lot of point in discussing it too much until someone shows signs of wanting to implement it :) One problem is how the API would model this nesting, another problem is making the scheduler aware that some nodes are only available to the tenant which owns them but maybe a bigger problem is the security model around allowing a node managed by an untrusted become a compute node. Mark. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
