Re: [Openstack] Limit flavors to specific hosts
Yes - Its more generic that hypervisor capabilities – my main problem with Host Aggregates is that it limits it to some specific 1:1 groupings based on hypervisor functionality. Use cases I want to be able to cover include: - Rolling new hardware through an existing cluster, and limiting some new flavors (which might for example provide higher network bandwidth) to just those servers - Providing a range of flavours that are dependent on specific hardware features (GPU) - There may be a further group that couples flavour and/or images to host groups – for example it’s possible to imagine a scenario where an image is only licensed to some specific subset of servers, or where a subset of nodes are running LXC (in which case the image is in effect pre-defined). In this case the image metadata could, for example, specify the flavors that it can be used with, and those flavors are in turn limited to specific hosts. I don’t really like this model of linking Glance objects (images) to Nova Objects (flavors), but I’m not sure what an alternative would be. On the config file vs REST API for configuration debate (maybe this needs to be a Design Summit subject in its own right), I agree that we should make a distinction between items that are deploy time configuration (which hypervisor to use, network driver, etc) and items that could change whilst the system is running (rate limits is a good example). I don’t however see this as being an REST API vs config file issue - more a configuration repository issue. I’d also add that anything which is going to be configured via a REST API needs to also provide a command line tool to drive that interface – so that out of the box the system can be installed and configured via the tools and scripts shipped with it. Phil From: openstack-bounces+philip.day=hp@lists.launchpad.net [mailto:openstack-bounces+philip.day=hp@lists.launchpad.net] On Behalf Of Jan Drake Sent: 03 April 2012 02:23 To: Lorin Hochstein Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Limit flavors to specific hosts If I understand this correctly, the motivation is to be able to provide a hint to schedulers on host-level appropriateness based on information external to that found in the hyperviser. Right/Wrong/Close? It would help to have a real-world example of where basic host resource evalution for scheduling would cause a situation requiring the host-level hard-coding of what is essentially a flavor-constraint. I'll hold further thoughts for downstream. Jan On Apr 2, 2012, at 6:06 PM, Lorin Hochstein lo...@nimbisservices.commailto:lo...@nimbisservices.com wrote: Just created a blueprint for this: https://blueprints.launchpad.net/nova/+spec/host-capabilities-api Take care, Lorin -- Lorin Hochstein Lead Architect - Cloud Services Nimbis Services, Inc. www.nimbisservices.comhttps://www.nimbisservices.com/ On Apr 2, 2012, at 3:29 PM, Jay Pipes wrote: Can I add a feature request to the below thoughtstream? Can we make it so that the management of these things can be done outside of config files? i.e. via a REST API with some simple middleware exposing the particular scheduler nodes' understanding of which capabilities/filters it is using to apply its scheduling algorithm? Making changes to configuration files works OK for simple uses and testing, not so much for on-demand operations :) I say this after grumbling about similar configuration obstacles with ratelimits. Best, -jay On 04/02/2012 02:37 PM, Chris Behrens wrote: I have some plans for being able to set arbitrary capabilities for hosts via nova.conf that you can use to build scheduler filters. Right now, there are capabilities, but I believe we're only creating these from hypervisor stats. You can filter on those today. What I'm planning on adding is a way to specify additional keyname/value pairs in nova.conf to supplement the capabilities we build from hypervisor stats. You could set things like this in your nova.conf: --host_capabilities=instance_type_ids=1,2,3;keyX;keyY=something etc. Since capabilities are already passed to scheduler rules, you could add some basic filters that do: if 'instance_type_ids' in capabilities and instance_type.id not in capabilities['instance_type_ids']: return False Since host_capabilities are just arbitrary keyname/value pairs, you can pretty much add anything you want to --host_capabilities and then write some matching scheduler filter rules. That's the basic idea, anyway. The exact same behavior will apply to 'cells' and the cells scheduler as well. (Except you'll have cells_capabilities= somewhere (prob nova.conf for the cells service). - Chris On Apr 2, 2012, at 10:36 AM, Day, Phil wrote: Hi Folks, I’m looking for a capability to limit some flavours to some hosts. I want the mapping to be as flexible as possible, and work within a zone/cell (I don’t want to add zones just to get
Re: [Openstack] Limit flavors to specific hosts
On Apr 3, 2012, at 6:45 AM, Day, Phil wrote: Hi John, Maybe the problem with host aggregates is that it too quickly became something that was linked to hypervisor capability, rather than being the more general mechanism of which one form of aggregate could be linked to hypervisor capabilities ? Should we have a “host aggregates 2.0” session at the Design Summit ? + 1 I think the primary use case is associating metadata with groups of hosts that can be interpreted by the scheduler. Obviously, this same metadata can be used to create pools/etc. in the hypervisor, but we can't forget about the scheduler. Modifying flags on the hosts for capabilities is ugly. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Limit flavors to specific hosts
+1 It is certainly worth a session to decide how to modify the scheduler. I suspect all you would need is to do on the compute side is add a stub implementations for add/remove host operations in the compute manager base class (rather than throw NotImplemented exceptions), and maybe an extra XenServer specific flag to suppress the pool creation. I think it has all the DB and metadata functionality required. One thing worth raising, is that currently all hosts in an aggregate must be in the same availability zone. That seems reasonable to me. From: Vishvananda Ishaya [mailto:vishvana...@gmail.com] Sent: 03 April 2012 19:11 To: Day, Phil Cc: John Garbutt; Jan Drake; Lorin Hochstein; openstack@lists.launchpad.net Subject: Re: [Openstack] Limit flavors to specific hosts On Apr 3, 2012, at 6:45 AM, Day, Phil wrote: Hi John, Maybe the problem with host aggregates is that it too quickly became something that was linked to hypervisor capability, rather than being the more general mechanism of which one form of aggregate could be linked to hypervisor capabilities ? Should we have a host aggregates 2.0 session at the Design Summit ? + 1 I think the primary use case is associating metadata with groups of hosts that can be interpreted by the scheduler. Obviously, this same metadata can be used to create pools/etc. in the hypervisor, but we can't forget about the scheduler. Modifying flags on the hosts for capabilities is ugly. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Limit flavors to specific hosts
+1. Interesting scenarios open up if we can have the scheduler intelligently direct workloads based on config/metadata. johnpur From: openstack-bounces+john=openstack@lists.launchpad.net [mailto:openstack-bounces+john=openstack@lists.launchpad.net] On Behalf Of Vishvananda Ishaya Sent: Tuesday, April 03, 2012 1:11 PM To: Day, Phil Cc: openstack@lists.launchpad.net; John Garbutt Subject: Re: [Openstack] Limit flavors to specific hosts On Apr 3, 2012, at 6:45 AM, Day, Phil wrote: Hi John, Maybe the problem with host aggregates is that it too quickly became something that was linked to hypervisor capability, rather than being the more general mechanism of which one form of aggregate could be linked to hypervisor capabilities ? Should we have a host aggregates 2.0 session at the Design Summit ? + 1 I think the primary use case is associating metadata with groups of hosts that can be interpreted by the scheduler. Obviously, this same metadata can be used to create pools/etc. in the hypervisor, but we can't forget about the scheduler. Modifying flags on the hosts for capabilities is ugly. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Limit flavors to specific hosts
On Tue, Apr 3, 2012 at 7:10 PM, Vishvananda Ishaya vishvana...@gmail.com wrote: On Apr 3, 2012, at 6:45 AM, Day, Phil wrote: Hi John, Maybe the problem with host aggregates is that it too quickly became something that was linked to hypervisor capability, rather than being the more general mechanism of which one form of aggregate could be linked to hypervisor capabilities ? The host aggregate model is not tied to no hypervisor-dependent detail, it's a group of hosts with metadata linked to it. As far as the the functionality around it, that can evolve to make it work as you expected it to work. If I understand it correctly, you'd not be interested in xenapi (whose implementation is wrapped around the concept of pool), and as far as KVM is concerned, this is still a blank canvas. As for the scheduler, as most things in Nova, you can write a new one (that's host-aggregate aware), or you can extend an existing one to make sure it understands the metadata related to a host belonging to an aggregate. The latter, in particular, was not done in the Essex timeframe for lack of time ;( Should we have a “host aggregates 2.0” session at the Design Summit ? I am all in for a host aggregate 2.0 so to speak, or just the continuation of the work that was started; at first I got the impression that we were going to write new abstraction/functionality from scratch just because the one that already existed did not fit the bill...I am glad I was wrong :) + 1 I think the primary use case is associating metadata with groups of hosts that can be interpreted by the scheduler. Obviously, this same metadata can be used to create pools/etc. in the hypervisor, but we can't forget about the scheduler. Modifying flags on the hosts for capabilities is ugly. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Limit flavors to specific hosts
Hi Folks, I'm looking for a capability to limit some flavours to some hosts. I want the mapping to be as flexible as possible, and work within a zone/cell (I don't want to add zones just to get this mapping).For example I want to express something like: Host_1 supports flavours A, C Host_2 supports flavours A, B Host_3 supports flavours A, B, C Host_4 supports flavours D Ideally there would be some form of grouping to sets of flavours: Flavour_A is part of Flavour_Sets 1, 2, 3 Flavour_B is part of Flavour_Sets 2, 3 Flavour_C is part of Flavour_Sets 1, 3, 4 Host_1 supports flavour Set 1 Host_2 supports flavour Set 2 Host_3 supports flavour Set 3 Host_4 supports flavour Set 4 From the Essex design summit I thought that host aggregates was going to give this sort of capability, but having looked through the code that seems to be quite tightly coupled with specific hypervisor functionality, whereas this is purely a scheduler feature. I can see that I could define flavour group membership through the instanace_type_extra_specs, but not how to then associate these with specific hosts. I know I'm a tad behind some of the recent changes - so before suggesting a design summit session on this I thought I'd ask - is there something that already does this type of mapping ? Cheers, Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Limit flavors to specific hosts
I have some plans for being able to set arbitrary capabilities for hosts via nova.conf that you can use to build scheduler filters. Right now, there are capabilities, but I believe we're only creating these from hypervisor stats. You can filter on those today. What I'm planning on adding is a way to specify additional keyname/value pairs in nova.conf to supplement the capabilities we build from hypervisor stats. You could set things like this in your nova.conf: --host_capabilities=instance_type_ids=1,2,3;keyX;keyY=something etc. Since capabilities are already passed to scheduler rules, you could add some basic filters that do: if 'instance_type_ids' in capabilities and instance_type.id not in capabilities['instance_type_ids']: return False Since host_capabilities are just arbitrary keyname/value pairs, you can pretty much add anything you want to --host_capabilities and then write some matching scheduler filter rules. That's the basic idea, anyway. The exact same behavior will apply to 'cells' and the cells scheduler as well. (Except you'll have cells_capabilities= somewhere (prob nova.conf for the cells service). - Chris On Apr 2, 2012, at 10:36 AM, Day, Phil wrote: Hi Folks, I’m looking for a capability to limit some flavours to some hosts. I want the mapping to be as flexible as possible, and work within a zone/cell (I don’t want to add zones just to get this mapping).For example I want to express something like: Host_1 supports flavours A, C Host_2 supports flavours A, B Host_3 supports flavours A, B, C Host_4 supports flavours D Ideally there would be some form of grouping to sets of flavours: Flavour_A is part of Flavour_Sets 1, 2, 3 Flavour_B is part of Flavour_Sets 2, 3 Flavour_C is part of Flavour_Sets 1, 3, 4 Host_1 supports flavour Set 1 Host_2 supports flavour Set 2 Host_3 supports flavour Set 3 Host_4 supports flavour Set 4 From the Essex design summit I thought that host aggregates was going to give this sort of capability, but having looked through the code that seems to be quite tightly coupled with specific hypervisor functionality, whereas this is purely a scheduler feature. I can see that I could define flavour group membership through the instanace_type_extra_specs, but not how to then associate these with specific hosts. I know I’m a tad behind some of the recent changes – so before suggesting a design summit session on this I thought I’d ask - is there something that already does this type of mapping ? Cheers, Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Limit flavors to specific hosts
Phil: I don't know of a way to do this in Essex. However, when I was working at ISI, we added a configuration option called extra_node_capabilities that allows you to specify the capabilities of a particular host in the /etc/nova/nova.conf, and these capabilities are checked against the extra_specs. See the following lines of code in ISI's repo: https://github.com/usc-isi/nova/blob/hpc-trunk/nova/virt/libvirt/connection.py#L163 https://github.com/usc-isi/nova/blob/hpc-trunk/nova/virt/libvirt/connection.py#L2669 I don't know how much reporting of capabilities has changed from diablo to essex, though, so no clue if that code would still work if ported to the current trunk. Take care, Lorin -- Lorin Hochstein Lead Architect - Cloud Services Nimbis Services, Inc. www.nimbisservices.com On Apr 2, 2012, at 1:36 PM, Day, Phil wrote: Hi Folks, I’m looking for a capability to limit some flavours to some hosts. I want the mapping to be as flexible as possible, and work within a zone/cell (I don’t want to add zones just to get this mapping).For example I want to express something like: Host_1 supports flavours A, C Host_2 supports flavours A, B Host_3 supports flavours A, B, C Host_4 supports flavours D Ideally there would be some form of grouping to sets of flavours: Flavour_A is part of Flavour_Sets 1, 2, 3 Flavour_B is part of Flavour_Sets 2, 3 Flavour_C is part of Flavour_Sets 1, 3, 4 Host_1 supports flavour Set 1 Host_2 supports flavour Set 2 Host_3 supports flavour Set 3 Host_4 supports flavour Set 4 From the Essex design summit I thought that host aggregates was going to give this sort of capability, but having looked through the code that seems to be quite tightly coupled with specific hypervisor functionality, whereas this is purely a scheduler feature. I can see that I could define flavour group membership through the instanace_type_extra_specs, but not how to then associate these with specific hosts. I know I’m a tad behind some of the recent changes – so before suggesting a design summit session on this I thought I’d ask - is there something that already does this type of mapping ? Cheers, Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Limit flavors to specific hosts
Can I add a feature request to the below thoughtstream? Can we make it so that the management of these things can be done outside of config files? i.e. via a REST API with some simple middleware exposing the particular scheduler nodes' understanding of which capabilities/filters it is using to apply its scheduling algorithm? Making changes to configuration files works OK for simple uses and testing, not so much for on-demand operations :) I say this after grumbling about similar configuration obstacles with ratelimits. Best, -jay On 04/02/2012 02:37 PM, Chris Behrens wrote: I have some plans for being able to set arbitrary capabilities for hosts via nova.conf that you can use to build scheduler filters. Right now, there are capabilities, but I believe we're only creating these from hypervisor stats. You can filter on those today. What I'm planning on adding is a way to specify additional keyname/value pairs in nova.conf to supplement the capabilities we build from hypervisor stats. You could set things like this in your nova.conf: --host_capabilities=instance_type_ids=1,2,3;keyX;keyY=something etc. Since capabilities are already passed to scheduler rules, you could add some basic filters that do: if 'instance_type_ids' in capabilities and instance_type.id not in capabilities['instance_type_ids']: return False Since host_capabilities are just arbitrary keyname/value pairs, you can pretty much add anything you want to --host_capabilities and then write some matching scheduler filter rules. That's the basic idea, anyway. The exact same behavior will apply to 'cells' and the cells scheduler as well. (Except you'll have cells_capabilities= somewhere (prob nova.conf for the cells service). - Chris On Apr 2, 2012, at 10:36 AM, Day, Phil wrote: Hi Folks, I’m looking for a capability to limit some flavours to some hosts. I want the mapping to be as flexible as possible, and work within a zone/cell (I don’t want to add zones just to get this mapping). For example I want to express something like: Host_1 supports flavours A, C Host_2 supports flavours A, B Host_3 supports flavours A, B, C Host_4 supports flavours D Ideally there would be some form of grouping to sets of flavours: Flavour_A is part of Flavour_Sets 1, 2, 3 Flavour_B is part of Flavour_Sets 2, 3 Flavour_C is part of Flavour_Sets 1, 3, 4 Host_1 supports flavour Set 1 Host_2 supports flavour Set 2 Host_3 supports flavour Set 3 Host_4 supports flavour Set 4 From the Essex design summit I thought that host aggregates was going to give this sort of capability, but having looked through the code that seems to be quite tightly coupled with specific hypervisor functionality, whereas this is purely a scheduler feature. I can see that I could define flavour group membership through the instanace_type_extra_specs, but not how to then associate these with specific hosts. I know I’m a tad behind some of the recent changes – so before suggesting a design summit session on this I thought I’d ask - is there something that already does this type of mapping ? Cheers, Phil ___ Mailing list:https://launchpad.net/~openstack Post to :openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe :https://launchpad.net/~openstack More help :https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Limit flavors to specific hosts
+1 -Original Message- From: openstack-bounces+ed_conzel=dell@lists.launchpad.net [mailto:openstack-bounces+ed_conzel=dell@lists.launchpad.net] On Behalf Of Jay Pipes Sent: Monday, April 02, 2012 2:29 PM To: openstack@lists.launchpad.net Subject: Re: [Openstack] Limit flavors to specific hosts Can I add a feature request to the below thoughtstream? Can we make it so that the management of these things can be done outside of config files? i.e. via a REST API with some simple middleware exposing the particular scheduler nodes' understanding of which capabilities/filters it is using to apply its scheduling algorithm? Making changes to configuration files works OK for simple uses and testing, not so much for on-demand operations :) I say this after grumbling about similar configuration obstacles with ratelimits. Best, -jay On 04/02/2012 02:37 PM, Chris Behrens wrote: I have some plans for being able to set arbitrary capabilities for hosts via nova.conf that you can use to build scheduler filters. Right now, there are capabilities, but I believe we're only creating these from hypervisor stats. You can filter on those today. What I'm planning on adding is a way to specify additional keyname/value pairs in nova.conf to supplement the capabilities we build from hypervisor stats. You could set things like this in your nova.conf: --host_capabilities=instance_type_ids=1,2,3;keyX;keyY=something etc. Since capabilities are already passed to scheduler rules, you could add some basic filters that do: if 'instance_type_ids' in capabilities and instance_type.id not in capabilities['instance_type_ids']: return False Since host_capabilities are just arbitrary keyname/value pairs, you can pretty much add anything you want to --host_capabilities and then write some matching scheduler filter rules. That's the basic idea, anyway. The exact same behavior will apply to 'cells' and the cells scheduler as well. (Except you'll have cells_capabilities= somewhere (prob nova.conf for the cells service). - Chris On Apr 2, 2012, at 10:36 AM, Day, Phil wrote: Hi Folks, I'm looking for a capability to limit some flavours to some hosts. I want the mapping to be as flexible as possible, and work within a zone/cell (I don't want to add zones just to get this mapping). For example I want to express something like: Host_1 supports flavours A, C Host_2 supports flavours A, B Host_3 supports flavours A, B, C Host_4 supports flavours D Ideally there would be some form of grouping to sets of flavours: Flavour_A is part of Flavour_Sets 1, 2, 3 Flavour_B is part of Flavour_Sets 2, 3 Flavour_C is part of Flavour_Sets 1, 3, 4 Host_1 supports flavour Set 1 Host_2 supports flavour Set 2 Host_3 supports flavour Set 3 Host_4 supports flavour Set 4 From the Essex design summit I thought that host aggregates was going to give this sort of capability, but having looked through the code that seems to be quite tightly coupled with specific hypervisor functionality, whereas this is purely a scheduler feature. I can see that I could define flavour group membership through the instanace_type_extra_specs, but not how to then associate these with specific hosts. I know I'm a tad behind some of the recent changes - so before suggesting a design summit session on this I thought I'd ask - is there something that already does this type of mapping ? Cheers, Phil ___ Mailing list:https://launchpad.net/~openstack Post to :openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe :https://launchpad.net/~openstack More help :https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Limit flavors to specific hosts
If I understand this correctly, the motivation is to be able to provide a hint to schedulers on host-level appropriateness based on information external to that found in the hyperviser. Right/Wrong/Close? It would help to have a real-world example of where basic host resource evalution for scheduling would cause a situation requiring the host-level hard-coding of what is essentially a flavor-constraint. I'll hold further thoughts for downstream. Jan On Apr 2, 2012, at 6:06 PM, Lorin Hochstein lo...@nimbisservices.com wrote: Just created a blueprint for this: https://blueprints.launchpad.net/nova/+spec/host-capabilities-api Take care, Lorin -- Lorin Hochstein Lead Architect - Cloud Services Nimbis Services, Inc. www.nimbisservices.com On Apr 2, 2012, at 3:29 PM, Jay Pipes wrote: Can I add a feature request to the below thoughtstream? Can we make it so that the management of these things can be done outside of config files? i.e. via a REST API with some simple middleware exposing the particular scheduler nodes' understanding of which capabilities/filters it is using to apply its scheduling algorithm? Making changes to configuration files works OK for simple uses and testing, not so much for on-demand operations :) I say this after grumbling about similar configuration obstacles with ratelimits. Best, -jay On 04/02/2012 02:37 PM, Chris Behrens wrote: I have some plans for being able to set arbitrary capabilities for hosts via nova.conf that you can use to build scheduler filters. Right now, there are capabilities, but I believe we're only creating these from hypervisor stats. You can filter on those today. What I'm planning on adding is a way to specify additional keyname/value pairs in nova.conf to supplement the capabilities we build from hypervisor stats. You could set things like this in your nova.conf: --host_capabilities=instance_type_ids=1,2,3;keyX;keyY=something etc. Since capabilities are already passed to scheduler rules, you could add some basic filters that do: if 'instance_type_ids' in capabilities and instance_type.id not in capabilities['instance_type_ids']: return False Since host_capabilities are just arbitrary keyname/value pairs, you can pretty much add anything you want to --host_capabilities and then write some matching scheduler filter rules. That's the basic idea, anyway. The exact same behavior will apply to 'cells' and the cells scheduler as well. (Except you'll have cells_capabilities= somewhere (prob nova.conf for the cells service). - Chris On Apr 2, 2012, at 10:36 AM, Day, Phil wrote: Hi Folks, I’m looking for a capability to limit some flavours to some hosts. I want the mapping to be as flexible as possible, and work within a zone/cell (I don’t want to add zones just to get this mapping). For example I want to express something like: Host_1 supports flavours A, C Host_2 supports flavours A, B Host_3 supports flavours A, B, C Host_4 supports flavours D Ideally there would be some form of grouping to sets of flavours: Flavour_A is part of Flavour_Sets 1, 2, 3 Flavour_B is part of Flavour_Sets 2, 3 Flavour_C is part of Flavour_Sets 1, 3, 4 Host_1 supports flavour Set 1 Host_2 supports flavour Set 2 Host_3 supports flavour Set 3 Host_4 supports flavour Set 4 From the Essex design summit I thought that host aggregates was going to give this sort of capability, but having looked through the code that seems to be quite tightly coupled with specific hypervisor functionality, whereas this is purely a scheduler feature. I can see that I could define flavour group membership through the instanace_type_extra_specs, but not how to then associate these with specific hosts. I know I’m a tad behind some of the recent changes – so before suggesting a design summit session on this I thought I’d ask - is there something that already does this type of mapping ? Cheers, Phil ___ Mailing list:https://launchpad.net/~openstack Post to :openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe :https://launchpad.net/~openstack More help :https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Limit flavors to specific hosts
Jan: Here are two use cases from when I was at ISI. In both cases, we wanted access to a machine with customized hardware to improve performance of certain applications: 1. GPUs. We wanted the users to be able to specify request a host that had GPU cards so they could run GPU-accelarated apps in their instances. 2. Shared-memory machine. We had an SGI UV machine with specific performance characteristics, and sometimes we wanted to specifically launch instances on that machine, even if the instance could fit on one of the regular x86 boxes we had. So we had an extra spec with something like system=uv. Take care, Lorin -- Lorin Hochstein Lead Architect - Cloud Services Nimbis Services, Inc. www.nimbisservices.com On Apr 2, 2012, at 9:23 PM, Jan Drake wrote: If I understand this correctly, the motivation is to be able to provide a hint to schedulers on host-level appropriateness based on information external to that found in the hyperviser. Right/Wrong/Close? It would help to have a real-world example of where basic host resource evalution for scheduling would cause a situation requiring the host-level hard-coding of what is essentially a flavor-constraint. I'll hold further thoughts for downstream. Jan On Apr 2, 2012, at 6:06 PM, Lorin Hochstein lo...@nimbisservices.com wrote: Just created a blueprint for this: https://blueprints.launchpad.net/nova/+spec/host-capabilities-api Take care, Lorin -- Lorin Hochstein Lead Architect - Cloud Services Nimbis Services, Inc. www.nimbisservices.com On Apr 2, 2012, at 3:29 PM, Jay Pipes wrote: Can I add a feature request to the below thoughtstream? Can we make it so that the management of these things can be done outside of config files? i.e. via a REST API with some simple middleware exposing the particular scheduler nodes' understanding of which capabilities/filters it is using to apply its scheduling algorithm? Making changes to configuration files works OK for simple uses and testing, not so much for on-demand operations :) I say this after grumbling about similar configuration obstacles with ratelimits. Best, -jay On 04/02/2012 02:37 PM, Chris Behrens wrote: I have some plans for being able to set arbitrary capabilities for hosts via nova.conf that you can use to build scheduler filters. Right now, there are capabilities, but I believe we're only creating these from hypervisor stats. You can filter on those today. What I'm planning on adding is a way to specify additional keyname/value pairs in nova.conf to supplement the capabilities we build from hypervisor stats. You could set things like this in your nova.conf: --host_capabilities=instance_type_ids=1,2,3;keyX;keyY=something etc. Since capabilities are already passed to scheduler rules, you could add some basic filters that do: if 'instance_type_ids' in capabilities and instance_type.id not in capabilities['instance_type_ids']: return False Since host_capabilities are just arbitrary keyname/value pairs, you can pretty much add anything you want to --host_capabilities and then write some matching scheduler filter rules. That's the basic idea, anyway. The exact same behavior will apply to 'cells' and the cells scheduler as well. (Except you'll have cells_capabilities= somewhere (prob nova.conf for the cells service). - Chris On Apr 2, 2012, at 10:36 AM, Day, Phil wrote: Hi Folks, I’m looking for a capability to limit some flavours to some hosts. I want the mapping to be as flexible as possible, and work within a zone/cell (I don’t want to add zones just to get this mapping). For example I want to express something like: Host_1 supports flavours A, C Host_2 supports flavours A, B Host_3 supports flavours A, B, C Host_4 supports flavours D Ideally there would be some form of grouping to sets of flavours: Flavour_A is part of Flavour_Sets 1, 2, 3 Flavour_B is part of Flavour_Sets 2, 3 Flavour_C is part of Flavour_Sets 1, 3, 4 Host_1 supports flavour Set 1 Host_2 supports flavour Set 2 Host_3 supports flavour Set 3 Host_4 supports flavour Set 4 From the Essex design summit I thought that host aggregates was going to give this sort of capability, but having looked through the code that seems to be quite tightly coupled with specific hypervisor functionality, whereas this is purely a scheduler feature. I can see that I could define flavour group membership through the instanace_type_extra_specs, but not how to then associate these with specific hosts. I know I’m a tad behind some of the recent changes – so before suggesting a design summit session on this I thought I’d ask - is there something that already does this type of mapping ? Cheers, Phil ___ Mailing list:https://launchpad.net/~openstack Post to :openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe
