Re: [Openstack] Potential filter scheduler enhancement
Phil, I agree with the suggestion. We wanted to instantiate a set of VMs whose hosts are located close each other. The current scheduler could not handle this use case since it needed to return optimal set of hosts at the same time. The first optimal host returned by the current scheduler may not be a part of the set of optimal hosts. We worked on the scheduler a little (in draft mode), and if you are interested in it, please let me know. Thanks, Joseph - Original Message - From: Vishvananda Ishaya vishvana...@gmail.com To: Phil Day philip@hp.com Cc: openstack@lists.launchpad.net (openstack@lists.launchpad.net) (openstack@lists.launchpad.net) openstack@lists.launchpad.net Sent: Thursday, January 3, 2013 12:43:02 PM Subject: Re: [Openstack] Potential filter scheduler enhancement I think this seems reasonable, although FYI, openstack-dev seems like a better place for emails like this. Vish On Jan 3, 2013, at 6:40 AM, Day, Phil philip@hp.com wrote: Hi Folks, and Happy New Year. In working with the Filter Scheduler I’m considering an enhancement to make the final host selection stage configurable. Whilst its sometimes fine to just pick the first host from the list of weighted hosts, the more general case is that I’d like to be able to have the scheduler pick one of the first N hosts on the weighted list. The specific use cases that I have in mind are: - On a large system there is very rarely a single ideal / optimal host for a particular instance to be placed on. In practice any of the N most suitable hosts would be fine and allowing the scheduler to randomly pick one of these would add some spread for multiple requests that come in at the same time. (I know we now have the retry mechanism if a particular host can’t in fact handle a specific request – this is a complement to that rather an alternative). Of course anyone who wants to schedule to host in strict weighted order would be able to configure N to be 1 (or we could keep the current host selection method as a separate default) - When creating M instances in one request we could just put each onto one of the first M hosts in the list (since these have all been filtered as being suitable) instead of having to iterate through the filter / weighting functions for each successive instance. Based on this I’m thinking that such a host_selection function would replace the whole of the for loop at the end of the _schedule() method in filter_scheduler.py, and take as input the number of instances. The default function would of course be the current behaviour. Before going any further with this thinking I wanted to get input on: i) Do others recognise these use cases as being useful, and are there other similar use cases to be considered at the same time ? ii) Is it reasonable to make the filter scheduler configurable in this way, rather than creating a separate scheduler ? (My feeling is that because it would only be replacing ~10% of the current filter_scheduler code it would be better to not create a new scheduler) iii) Should the configuration logic for this new function be in the fliter_scheduler itself, or in the host_manager (which is where the filter and weighting functions are configured) ? Cheers, Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-hpc] [HPC] Reminder monthly telecon Sep. 10
All, I have a blue print on proximity scheduler at http://wiki.openstack.org/ProximityScheduler, and would like to get feedback on it. Thanks, Joseph - Original Message - From: John Paul Walters jwalt...@isi.edu To: openstack@lists.launchpad.net Cc: openstack-...@lists.openstack.org Sent: Friday, September 7, 2012 12:12:20 PM Subject: [openstack-hpc] [HPC] Reminder monthly telecon Sep. 10 Hi, This is a reminder that we'll hold our next monthly HPC telecon this coming Monday, Sep. 10 and 12:00 noon Eastern Time. We'll use webex (details below). The agenda is somewhat open. Our default will be to start the conversation about HPC features that folks are interested in adding to the Grizzly release. If anyone has any other specific agenda items, they're welcome to propose them. I'm unable to attend, so my colleague David Kang will be hosting this meeting. We look forward to talking to you! best, JP John Paul Walters invites you to attend this online meeting. Topic: HPC Monthly Telecon Date: Monday, September 10, 2012 Time: 12:00 pm, Eastern Daylight Time (New York, GMT-04:00) Meeting Number: 927 246 497 Meeting Password: hpcmonthly --- To join the online meeting (Now from mobile devices!) --- 1. Go to https://openstack.webex.com/openstack/j.php?ED=203524102UID=1431607857PW=NYzljOTEwYThjRT=MiMxMQ%3D%3D 2. If requested, enter your name and email address. 3. If a password is required, enter the meeting password: hpcmonthly 4. Click Join. To view in other time zones or languages, please click the link: https://openstack.webex.com/openstack/j.php?ED=203524102UID=1431607857PW=NYzljOTEwYThjORT=MiMxMQ%3D%3D ___ OpenStack-HPC mailing list openstack-...@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-hpc ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Question regarding who the nova database is updated
Heng, I maybe wrong as the latest code is somewhat different from the one I followed last time, but it looks like the information is update at line 440 in nova/nova/compute/resource_tracker.py. Thanks, Joseph - Original Message - From: Heng Xu shouhengzhang...@mail.utoronto.ca To: openstack@lists.launchpad.net Sent: Tuesday, August 28, 2012 1:52:18 PM Subject: [Openstack] Question regarding who the nova database is updated Hi folks: I was trying to add a new column to the Nova database table compute_nodes. however, I was not sure of how it is updated periodically, all I know is that comsume_from_instance() function in host_manager.py update the memory, then I was not able to find how nova subsequently update the compute_nodes table in the database. Any help will be appreciated, please let me know the exact location (file, function, etc.) and how I can mimc that to modify my own new field in the compute_nodes table. Thanks in advance. Heng ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
A bug fix patch that adds the extra specs matching against host aggregate is under review at https://review.openstack.org/#/c/11449/. Thanks, Joseph - Original Message - From: Vishvananda Ishaya vishvana...@gmail.com To: Joseph Suh j...@isi.edu Cc: Patrick Petit patrick.michel.pe...@gmail.com, openstack@lists.launchpad.net (openstack@lists.launchpad.net) openstack@lists.launchpad.net Sent: Friday, August 24, 2012 10:10:05 PM Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications Folsom also supports setting key values for things like capabilities via host aggregates. There is a filter[1] that matches the extra specs by exact comparison just like was done for capabilities before the last patch. The new extra specs matching should be added to it. These capabilities can be set dynamically by administrators so it directly supports the use case below. [1] https://github.com/openstack/nova/blob/master/nova/scheduler/filters/aggregate_instance_extra_specs.py On Aug 24, 2012, at 6:39 PM, Joseph Suh j...@isi.edu wrote: Patrick, That's a good point. I think the issue is already being discussed at https://bugs.launchpad.net/nova/+bug/1039386 as Don Dugger pointed out. That being said, as answers to some of your questions: yes, any key/value pair can be used and it is user's (in this case, system admin's) responsibility to avoid conflict at this time. The scope we originally thought was pretty much static like the number of GPUs, but there is no reason why it should be static as some features can change dynamically. I'd encourage you to propose a blueprint if you can. We can also consider the feature, but our team needs to discuss it before we can commit to it. Thanks, Joseph - Original Message - From: Patrick Petit patrick.michel.pe...@gmail.com To: Joseph Suh j...@isi.edu Cc: openstack@lists.launchpad.net (openstack@lists.launchpad.net) openstack@lists.launchpad.net, David Kang dk...@isi.edu Sent: Friday, August 24, 2012 7:37:31 PM Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications Hi Joseph and All, You are pointing the root cause of my question. The question is about how to specify capabilities for a compute node so that they can be compared with the extra specs. I think how to define extra specs in a flavor is clear enough. So, some capabilities are standard and are generated dynamically. Others are not known or not captured by the system and so not standard (yet), like the GPUs, and therefore must be specified somehow. Today, the somehow, as I understand things, is to add key/value pairs in nova.conf when/if it is supported. I wanted to make sure i understand the basic principals. Now, this in my opinion poses couple problems and/or call for additional questions: What is the naming convention to add capabilities in nova.conf? I would suppose that any key/value pair cannot be taken for a capability. How to avoid name clashing with standard capability? At the very least one should have an option to print them out (in nova-manage?). Even a simple written list would be helpful. But, are we really comfortable with the idea to define static capabilities in nova.conf? that's putting a lot of burden on config management. Also, not standard doesn't imply static. We can certainly live we that for now, But eventually, i think we'll need some sort of an extension mechanism so that providers can generate whatever capabilities they want using their own plugin? Note that capabilities could be software related too. What do you think? Best, Patrick Envoyé de mon iPad Le 24 août 2012 à 18:38, Joseph Suh j...@isi.edu a écrit : Patrick, Once a new item (key and value pair) is added to the capabilities, it can be compared against extra_specs. The extra_specs can be populated in instance_type_extra_specs table. The items in the extra_specs can start with one of the keywords for operations such as = and s==. For example, if ngpus: 4 is populated in capability, extra_specs of = 2 will choose the host. If the extra_specs is = 5, the host will not be chosen. If no keyword is found at the beginning of the extra_specs (with the latest change in upstream code), the given string is directly compared with capability. For example, if fpu is given as extra_specs, the capability must be fpu to be selected. If more clarification is needed, please let us know. Thanks, Joseph - Original Message - From: David Kang dk...@isi.edu To: Patrick Petit patrick.michel.pe...@gmail.com Cc: openstack@lists.launchpad.net (openstack@lists.launchpad.net) openstack@lists.launchpad.net Sent: Friday, August 24, 2012 11:34:11 AM Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications Parick, We are using the feature in Bare-metal machine provisioning. Some keys are automatically generated by nova-compute
Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
Patrick, Once a new item (key and value pair) is added to the capabilities, it can be compared against extra_specs. The extra_specs can be populated in instance_type_extra_specs table. The items in the extra_specs can start with one of the keywords for operations such as = and s==. For example, if ngpus: 4 is populated in capability, extra_specs of = 2 will choose the host. If the extra_specs is = 5, the host will not be chosen. If no keyword is found at the beginning of the extra_specs (with the latest change in upstream code), the given string is directly compared with capability. For example, if fpu is given as extra_specs, the capability must be fpu to be selected. If more clarification is needed, please let us know. Thanks, Joseph - Original Message - From: David Kang dk...@isi.edu To: Patrick Petit patrick.michel.pe...@gmail.com Cc: openstack@lists.launchpad.net (openstack@lists.launchpad.net) openstack@lists.launchpad.net Sent: Friday, August 24, 2012 11:34:11 AM Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications Parick, We are using the feature in Bare-metal machine provisioning. Some keys are automatically generated by nova-compute. For example, hypervisor_type, hypervisor_version, etc. fields are automatically put into capabilities by nova-compute (in the case of libvirt). So, you don't need to specify that. But, if you want to add custom fields, you should put them into nova.conf file of the nova-compute node. Since the new key are put into 'capabilities', the new key must be different from any other keys in the 'capabilities'. If that uniqueness is enforced, it can be any string, I believe. Thanks, David -- Dr. Dong-In David Kang Computer Scientist USC/ISI - Original Message - Hi, Could someone give a practical overview of how configuring and using the instance type extra specs extension capability introduced in Folsom? If how extending an instance type is relatively clear. Eg.: #nova-manage instance_type set_key --name=my.instancetype --key cpu_arch --value 's== x86_64' The principles of capability advertising is less clearer. Is it assumed that the key/value pairs are always declared statically as flags in nova.conf of the compute node, or can they be generated dynamically and if so, who would that be? And also, are the keys completely free form strings or strings that are known (reserved) by Nova? Thanks in advance for clarifying this. Patrick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
Patrick, That's a good point. I think the issue is already being discussed at https://bugs.launchpad.net/nova/+bug/1039386 as Don Dugger pointed out. That being said, as answers to some of your questions: yes, any key/value pair can be used and it is user's (in this case, system admin's) responsibility to avoid conflict at this time. The scope we originally thought was pretty much static like the number of GPUs, but there is no reason why it should be static as some features can change dynamically. I'd encourage you to propose a blueprint if you can. We can also consider the feature, but our team needs to discuss it before we can commit to it. Thanks, Joseph - Original Message - From: Patrick Petit patrick.michel.pe...@gmail.com To: Joseph Suh j...@isi.edu Cc: openstack@lists.launchpad.net (openstack@lists.launchpad.net) openstack@lists.launchpad.net, David Kang dk...@isi.edu Sent: Friday, August 24, 2012 7:37:31 PM Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications Hi Joseph and All, You are pointing the root cause of my question. The question is about how to specify capabilities for a compute node so that they can be compared with the extra specs. I think how to define extra specs in a flavor is clear enough. So, some capabilities are standard and are generated dynamically. Others are not known or not captured by the system and so not standard (yet), like the GPUs, and therefore must be specified somehow. Today, the somehow, as I understand things, is to add key/value pairs in nova.conf when/if it is supported. I wanted to make sure i understand the basic principals. Now, this in my opinion poses couple problems and/or call for additional questions: What is the naming convention to add capabilities in nova.conf? I would suppose that any key/value pair cannot be taken for a capability. How to avoid name clashing with standard capability? At the very least one should have an option to print them out (in nova-manage?). Even a simple written list would be helpful. But, are we really comfortable with the idea to define static capabilities in nova.conf? that's putting a lot of burden on config management. Also, not standard doesn't imply static. We can certainly live we that for now, But eventually, i think we'll need some sort of an extension mechanism so that providers can generate whatever capabilities they want using their own plugin? Note that capabilities could be software related too. What do you think? Best, Patrick Envoyé de mon iPad Le 24 août 2012 à 18:38, Joseph Suh j...@isi.edu a écrit : Patrick, Once a new item (key and value pair) is added to the capabilities, it can be compared against extra_specs. The extra_specs can be populated in instance_type_extra_specs table. The items in the extra_specs can start with one of the keywords for operations such as = and s==. For example, if ngpus: 4 is populated in capability, extra_specs of = 2 will choose the host. If the extra_specs is = 5, the host will not be chosen. If no keyword is found at the beginning of the extra_specs (with the latest change in upstream code), the given string is directly compared with capability. For example, if fpu is given as extra_specs, the capability must be fpu to be selected. If more clarification is needed, please let us know. Thanks, Joseph - Original Message - From: David Kang dk...@isi.edu To: Patrick Petit patrick.michel.pe...@gmail.com Cc: openstack@lists.launchpad.net (openstack@lists.launchpad.net) openstack@lists.launchpad.net Sent: Friday, August 24, 2012 11:34:11 AM Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications Parick, We are using the feature in Bare-metal machine provisioning. Some keys are automatically generated by nova-compute. For example, hypervisor_type, hypervisor_version, etc. fields are automatically put into capabilities by nova-compute (in the case of libvirt). So, you don't need to specify that. But, if you want to add custom fields, you should put them into nova.conf file of the nova-compute node. Since the new key are put into 'capabilities', the new key must be different from any other keys in the 'capabilities'. If that uniqueness is enforced, it can be any string, I believe. Thanks, David -- Dr. Dong-In David Kang Computer Scientist USC/ISI - Original Message - Hi, Could someone give a practical overview of how configuring and using the instance type extra specs extension capability introduced in Folsom? If how extending an instance type is relatively clear. Eg.: #nova-manage instance_type set_key --name=my.instancetype --key cpu_arch --value 's== x86_64' The principles of capability advertising is less clearer. Is it assumed that the key/value pairs are always declared statically as flags in nova.conf of the compute node, or can they be generated
Re: [Openstack] Unable to start nova-scheduler : duplicate option: scheduler_host_manager
Ben, As the error message suggests, it is due to a duplicated option of scheduler_host_manager. It is specified more than once somewhere. Thanks, Joseph - Original Message - From: Ben ben@gmail.com To: openstack@lists.launchpad.net Sent: Wednesday, August 22, 2012 8:27:39 AM Subject: Re: [Openstack] Unable to start nova-scheduler : duplicate option: scheduler_host_manager Hi, No idea for my scheduler problem ? It was working at the beginning (I suppose because I could launch instances), but I did a lot of modification, modifying networks, deleting nova db, rebuild it, etc. Any idea on what I can do to identify the problem ? Where can I find the mentionned option except in nova.conf ? Ben Le 22/08/2012 00:46, Ben a écrit : Hi, I'm trying to setup a little nova cluster with 3 nodes : - 1 controller node running all services but compute - 2 compute nodes running compute and network only I have faced a lot of issues, but I can't understand this one. When I start nova-scheduler on controller node, the process dies instantly with this error : CRITICAL nova [-] duplicate option: scheduler_host_manager So I can't start an instance, it remains stuck in building state. I have checked my nova.conf file, and I only see this line : --scheduler_driver=nova.scheduler.simple.SimpleScheduler What does means this error, how can I solve it ? Thanks, Ben ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Unable to start nova-scheduler : duplicate option: scheduler_host_manager
Ben, It is possible to have the option specified in a code. Try grep -r on whole code. Thanks, Joseph - Original Message - From: Ben ben@gmail.com To: Joseph Suh j...@isi.edu Cc: openstack@lists.launchpad.net Sent: Wednesday, August 22, 2012 10:47:35 AM Subject: Re: [Openstack] Unable to start nova-scheduler : duplicate option: scheduler_host_manager Hi Joseph, Thank you for your answer. Yes, but where could be that option ? It's not duplicated in my nova.conf file, and this is the file the nova-scheduler uses. My nova.conf file joined. Regards, Ben Le 22/08/2012 16:37, Joseph Suh a écrit : Ben, As the error message suggests, it is due to a duplicated option of scheduler_host_manager. It is specified more than once somewhere. Thanks, Joseph - Original Message - From: Ben ben@gmail.com To: openstack@lists.launchpad.net Sent: Wednesday, August 22, 2012 8:27:39 AM Subject: Re: [Openstack] Unable to start nova-scheduler : duplicate option: scheduler_host_manager Hi, No idea for my scheduler problem ? It was working at the beginning (I suppose because I could launch instances), but I did a lot of modification, modifying networks, deleting nova db, rebuild it, etc. Any idea on what I can do to identify the problem ? Where can I find the mentionned option except in nova.conf ? Ben Le 22/08/2012 00:46, Ben a écrit : Hi, I'm trying to setup a little nova cluster with 3 nodes : - 1 controller node running all services but compute - 2 compute nodes running compute and network only I have faced a lot of issues, but I can't understand this one. When I start nova-scheduler on controller node, the process dies instantly with this error : CRITICAL nova [-] duplicate option: scheduler_host_manager So I can't start an instance, it remains stuck in building state. I have checked my nova.conf file, and I only see this line : --scheduler_driver=nova.scheduler.simple.SimpleScheduler What does means this error, how can I solve it ? Thanks, Ben ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using extra specs
Boris-Michel,
One thing that I noticed was a typo: schedulre that can cause malfunction. I am
not sure what version you are using, but recently the extra_spec checking is
moved to compute_capabilities_filter.py (ComputeCapabilitiesFilter). As far as
I understand, the current ComputeFilter does not check extra_specs.
Thanks,
Joseph
(w) 703-248-6160
(c) 571-340-2434
(f) 703-812-3712
http://www.east.isi.edu/~jsuh
Information Sciences Institute
University of Southern California
3811 N. Fairfax Drive Suite 200
Arlington, VA, 22203, USA
- Original Message -
From: Boris-Michel Deschenes boris-michel.desche...@ubisoft.com
To: openstack@lists.launchpad.net
Sent: Thursday, August 9, 2012 11:10:15 AM
Subject: [Openstack] using extra specs
Hi guys,
I currently have a working cloud with a working GPU passthrough setup
(CentOS/libvirt/Xen 4.1.2), now I need to work on adding this new resource to
openstack.
Here is the plan:
1. Create a new instance type (g1.small) with an extra spec like “xpu_arch =
radeon”
2. Modify the compute side so that nova-compute publishes this new capability
I’m in between step 1 and step 2, so normally, when I try and launch a VM with
the new instance type, the scheduling should fail since there is no compute
node publishing this capability yet, but the scheduling does go through despite
having the ComputeFilter available.
My understanding is that by having:
scheduler_available_filters=nova.schedulre.filters.standard_filters
scheduler_default_filters=ComputeFilter
The ComputeFilter should try and ensure that the hosts satisfies the extra
specs but the filter is not doing its job since the VM is scheduler and casted
to the compute node although it does not advertise any of the extra spec.
The extra spec is definitely part of the instance_type, I can see it in the
scheduler log:
(TRUNCATED) u'8895af5063b176bef97ab81f076d57f3', u'min_disk': u'0',
u'is_public': True, u'deleted_at': None, u'properties': {u'os_type':
u'windows'}, u'size': 9256562688}, u'instance_type': {u'root_gb': 0,
u'deleted_at': None, u'name': u'g1.small', u'deleted': False, u'created_at':
u'2012-08-08 18:33:29', u'ephemeral_gb': 0, u'updated_at': None, u'memory_mb':
2048, u'vcpus': 1, u'extra_specs': {u'xpu_arch': u'radeon'}, u'swap': 0,
u'rxtx_factor': 1.0, u'flavorid': u'105', u'vcpu_weight': None, u'id': 6},
u'instance_properties': {u'vm_state': u'building', u'availability_zone':
(TRUNCATED)
So again, this VM should not be casted to any compute node and the scheduler
should complain about no host having the required capability but it does not…
the VM is instantiated on the one compute node even though it never published
the required capability.
It looks like the ComputeFilter is either not used properly or I’m missing
something. Basically I want the scheduler to fail when I launch the VM before I
start working on step 2.
Any help appreciated.
Boris
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cannot pass hint to Nova Scheduler
Heng, You can print the values in the HostState class. If you want to monitor the changes of the values, for example, you can print (either directly or using LOG.debug()), the values in the code where you want monitor, for example, consume_from_instance(). Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: Jay Pipes jaypi...@gmail.com To: Heng Xu shouhengzhang...@mail.utoronto.ca Cc: openstack@lists.launchpad.net Sent: Monday, August 6, 2012 12:11:58 PM Subject: Re: [Openstack] Cannot pass hint to Nova Scheduler On 08/04/2012 07:48 AM, Heng Xu wrote: But I tried a few things in HostState class, I ran into error, because I could not monitor the stats in hoststates class as opposed to a database, is there a way to check the stats in HostState class as exists in memory? cc'ing Sandy Walsh, who is vastly more familiar with the scheduler than I am :) Sandy, see Heng's question above... seems like a great question to me -- and also a possible mini-project for someone to work on that would add a scheduler-diagnostics extension if such functionality isn't readily available. Best, -jay Heng From: Jay Pipes [jaypi...@gmail.com] Sent: Friday, August 03, 2012 4:38 PM To: Heng Xu Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Cannot pass hint to Nova Scheduler On 08/03/2012 09:28 AM, Heng Xu wrote: Another questions is, I can get all the status of a computing node in the mysql nova database, and select * from compute_node, but now I am using json filter, the only field I have success with now is the free_ram_db, if my hint uses free_disk_gb, then I always get error, but the database is showing my compute node has $free_disk_gb equal 17, so I was wondering, where to find exactly what kind of json field can use in json filter, thanks in advance The nova.scheduler.host_manager.HostState class is what is checked for attributes, not the ComputeNode model. So, you need to use $free_disk_mb, not free_disk_gb. Best, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] One question on the compute_filter
Yunhong, Thanks for your interest in our patch. The original purpose of the instance_type_extra_specs is providing ability to specify any specs that cannot be enumerated at the nova coding time such as hardware-specific requirements (like gpu) or any features in the future system (like python version 10). System administrator will populate necessary items at the time of deployment as he/she wants. Current compute_filter is a relatively simple code that compares the items in the instance_type_extra_specs with those in capability. In our patch, we want to add more operators, not changing the basic behavior. The idea behind the compute_filter is that if items are specified in instance_type_extra_specs, the filter makes it sure that all the requested items are in the capability of the provisioned hosts. So, in your approach, if you already know items that is required such as xpu_arch and want to ignore other items in the instance_type_extra_spec, one way might be to create another filter that checks only those known items. But, then, if there is a need to check another item or ignore an item in the future, the filter code needs to be modified. In our approach, it can be simply added/deleted in database by system administrator. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: Yunhong Jiang yunhong.ji...@intel.com To: j...@isi.edu Cc: Donald D Dugger donald.d.dug...@intel.com, openstack@lists.launchpad.net Sent: Monday, July 16, 2012 10:00:44 AM Subject: One question on the compute_filter Hi, Joseph I’m working on the patch for blueprints https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value , to add/delete the extra_specs for flavor through nova-manage. I’m still setting up my environment to push the patch. However, when I was testing my patch, I noticed that compute_filter assume it will handle all of the “extra_specs”. If it can’t find corresponding key in the capabilities, it will fail to pass the host. IMHO, this is a bit overkill. For example, currently the trusted_filter.py will use the extra_specs to check if trusted_host is required, that means compute filter and trusted filter can’t be used at the same time. I think compute filter should define explicitly all keys that it takes care, like cpu_info, cpu_arch, xpu_arch, and only check the corresponding extra_specs key/value pair? After all, extra_specs is not compute_extra_specs? I noticed the patch in https://review.openstack.org/#/c/8089/ , but seems this patch will not fix this issue still. Any idea or suggestion? I’m glad to create patch if any conclusion on this issue. Thanks --jyh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] One question on the compute_filter
Yunhong, I understand your concern. It has a different purpose than the compute_filter, so it has its own merit and can co-exist. The question is how much the demand is... Thanks, Joseph (w) 703-248-6160 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: Yunhong Jiang yunhong.ji...@intel.com To: Joseph Suh j...@isi.edu Cc: Donald D Dugger donald.d.dug...@intel.com, openstack@lists.launchpad.net Sent: Tuesday, July 17, 2012 11:39:54 AM Subject: RE: One question on the compute_filter -Original Message- From: Joseph Suh [mailto:j...@isi.edu] Sent: Tuesday, July 17, 2012 9:38 PM To: Jiang, Yunhong Cc: Dugger, Donald D; openstack@lists.launchpad.net Subject: Re: One question on the compute_filter Yunhong, Thanks for your interest in our patch. The original purpose of the instance_type_extra_specs is providing ability to specify any specs that cannot be enumerated at the nova coding time such as hardware-specific requirements (like gpu) or any features in the future system (like python version 10). System administrator will populate necessary items at the time of deployment as he/she wants. Current compute_filter is a relatively simple code that compares the items in the instance_type_extra_specs with those in capability. In our patch, we want to add more operators, not changing the basic behavior. Joseph, thanks for clarification. The idea behind the compute_filter is that if items are specified in instance_type_extra_specs, the filter makes it sure that all the requested items are in the capability of the provisioned hosts. So, in your approach, if you already know items that is required such as xpu_arch and want to ignore other items in the instance_type_extra_spec, one way might be to create another filter that checks only those known items. But, The issue here is, if I create such another filter, then it possibly can't co-exist with compute filter :( --jyh then, if there is a need to check another item or ignore an item in the future, the filter code needs to be modified. In our approach, it can be simply added/deleted in database by system administrator. Thanks, Joseph ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] One question on the compute_filter
Don, That's an interesting idea, but I am having a difficulty in understanding why you want to store extra information (not spec) in instance_type_extra_specs. If you need to keep some extra information, using or creating another flag or field in database might be a better place for that? The intention of having instance_type_extra_specs is to extend the instance_types table to accommodate extra specs that cannot be enumerated at the time of coding, and it would be best if we have the same behavior as instance_types in my opinion. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: Donald D Dugger donald.d.dug...@intel.com To: Joseph Suh j...@isi.edu, Yunhong Jiang yunhong.ji...@intel.com Cc: openstack@lists.launchpad.net Sent: Tuesday, July 17, 2012 11:47:25 AM Subject: RE: One question on the compute_filter Joeseph- My concern is that this means that the compute filter consumes `every` entry in the `extra_specs' table. I can imagine scenarios where other filters would want to put data in `extra_specs' that is not intended for the compute filter. In fact, we do that today with the trusted filter that was recently added where there is a `trusted_host' key that is added to the `extra_specs' table. I really like the idea of adding operators to the compute filter. This would mean that if you truly wanted the compute filter to check for the presence of capability you just explicitly specify the equality operator. Any `extra_specs' keys that don't have an operator would just be ignored, giving the administrator maximum flexibility in specifying capabilities. -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 -Original Message- From: Joseph Suh [mailto:j...@isi.edu] Sent: Tuesday, July 17, 2012 7:38 AM To: Jiang, Yunhong Cc: Dugger, Donald D; openstack@lists.launchpad.net Subject: Re: One question on the compute_filter Yunhong, Thanks for your interest in our patch. The original purpose of the instance_type_extra_specs is providing ability to specify any specs that cannot be enumerated at the nova coding time such as hardware-specific requirements (like gpu) or any features in the future system (like python version 10). System administrator will populate necessary items at the time of deployment as he/she wants. Current compute_filter is a relatively simple code that compares the items in the instance_type_extra_specs with those in capability. In our patch, we want to add more operators, not changing the basic behavior. The idea behind the compute_filter is that if items are specified in instance_type_extra_specs, the filter makes it sure that all the requested items are in the capability of the provisioned hosts. So, in your approach, if you already know items that is required such as xpu_arch and want to ignore other items in the instance_type_extra_spec, one way might be to create another filter that checks only those known items. But, then, if there is a need to check another item or ignore an item in the future, the filter code needs to be modified. In our approach, it can be simply added/deleted in database by system administrator. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: Yunhong Jiang yunhong.ji...@intel.com To: j...@isi.edu Cc: Donald D Dugger donald.d.dug...@intel.com, openstack@lists.launchpad.net Sent: Monday, July 16, 2012 10:00:44 AM Subject: One question on the compute_filter Hi, Joseph I’m working on the patch for blueprints https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value , to add/delete the extra_specs for flavor through nova-manage. I’m still setting up my environment to push the patch. However, when I was testing my patch, I noticed that compute_filter assume it will handle all of the “extra_specs”. If it can’t find corresponding key in the capabilities, it will fail to pass the host. IMHO, this is a bit overkill. For example, currently the trusted_filter.py will use the extra_specs to check if trusted_host is required, that means compute filter and trusted filter can’t be used at the same time. I think compute filter should define explicitly all keys that it takes care, like cpu_info, cpu_arch, xpu_arch, and only check the corresponding extra_specs key/value pair? After all, extra_specs is not compute_extra_specs? I noticed the patch in https://review.openstack.org/#/c/8089/ , but seems this patch will not fix this issue still. Any idea or suggestion? I’m glad to create patch if any conclusion on this issue. Thanks --jyh
Re: [Openstack] One question on the compute_filter
Don, Hmm, so the use of the instance_type_extra_specs is not for specifications purely any more... I am open to your proposed idea that if there is no operator, it will ignore the item (that is not original behavior of compute_filter, though). So, let me wait and see if there are any objections/comments on this issue from community, and if I don't hear them, I will implement my patch in that way. Thanks, Joseph (w) 703-248-6160 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: Donald D Dugger donald.d.dug...@intel.com To: Joseph Suh j...@isi.edu, Sandy Walsh sandy.wa...@rackspace.com, Vishvananda Ishaya vishvana...@gmail.com Cc: openstack@lists.launchpad.net, Yunhong Jiang yunhong.ji...@intel.com Sent: Tuesday, July 17, 2012 12:19:35 PM Subject: RE: One question on the compute_filter Joseph- Basically, Vish Sandy convinced me that the `instance_type_extra_specs' was the right place to put the trust info, rather than creating a new table. I like the idea of expanding the idea of extra_specs to be used to store extra information. I can easily imagine cases where someone wants to create a new, specialized filter that requires more information (the trusted filter is one example, there could be others). Especially if we want the ability to add filters that aren't part of the `official` source tree, and therefore can’t change standard APIs, you'll need a place to store more info and the `instance_type_extra_specs' table is the obvious place. Adding in comparison operators, especially the equality operator, would make it easy to have all the filters co-exist. -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack] duplicate option: scheduler_host_manager
Leander, As the error message indicates, it usually comes when the option is found in more than one place. Thanks, Joseph (w) 703-248-6160 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: Leander Bessa Beernaert leande...@gmail.com To: openstack@lists.launchpad.net Sent: Friday, June 29, 2012 11:45:49 AM Subject: [Openstack] [OpenStack] duplicate option: scheduler_host_manager Hello, I'm working on this patch https://review.openstack.org/#/c/8839/ and i keep running into this error https://jenkins.openstack.org/job/gate-nova-python26/1333/testReport/%3Cnose.suite/ContextSuite%20context=nova/tests__setup/ Can someone shed some light on the subject? Regards, Leander ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Jenkins
Gary, Apparently, it happened to me. Tests took 13 hours and came with failure. Thanks, Joseph (w) 703-248-6160 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: Gary Kotton gkot...@redhat.com To: openstack@lists.launchpad.net Sent: Wednesday, June 27, 2012 1:04:28 PM Subject: [Openstack] Jenkins Hi, Is anyone aware of a problem with Jenkins? Thanks Gary ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] No tests available in custom branch
Leander, Did you give -V -f option? If you post your log, it would be easier to help. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: Leander Bessa Beernaert leande...@gmail.com To: openstack@lists.launchpad.net Sent: Monday, June 25, 2012 9:13:43 AM Subject: [Openstack] [OpenStack][Nova] No tests available in custom branch Hello, I've created a separate branch in Nova to test my code out. However, every time i attempt to run the command run_tests.sh it says it has 0 tests to be run. I followed the guide from http://wiki.openstack.org/GerritWorkflow to setup the branch. Did i miss something? Regards, Leander ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] No tests available in custom branch
Leander, If it works fine in master branch, the problem must be due to some changes in your own branch. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: Leander Bessa Beernaert leande...@gmail.com To: Joseph Suh j...@isi.edu Cc: openstack@lists.launchpad.net Sent: Monday, June 25, 2012 9:45:51 AM Subject: Re: [Openstack] [OpenStack][Nova] No tests available in custom branch I've tried all the options and the output is always, even with -V and -f: -- Ran 0 tests in 0.000s OK Running PEP8 and HACKING compliance check... 5 imports missing in this test environment The tests run just fine on the master branch :/ On Mon, Jun 25, 2012 at 2:39 PM, Joseph Suh j...@isi.edu wrote: Leander, Did you give -V -f option? If you post your log, it would be easier to help. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: Leander Bessa Beernaert leande...@gmail.com To: openstack@lists.launchpad.net Sent: Monday, June 25, 2012 9:13:43 AM Subject: [Openstack] [OpenStack][Nova] No tests available in custom branch Hello, I've created a separate branch in Nova to test my code out. However, every time i attempt to run the command run_tests.sh it says it has 0 tests to be run. I followed the guide from http://wiki.openstack.org/GerritWorkflow to setup the branch. Did i miss something? Regards, Leander ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Testing NOVA-OVS-Quantum setup
Trinath, I found the following Quantun admin guide was useful for that purpose: http://www.google.com/url?sa=trct=jq=esrc=ssource=webcd=4ved=0CHsQFjADurl=http%3A%2F%2Fdocs.openstack.org%2Ftrunk%2Fopenstack-network%2Fadmin%2Fquantum-admin-guide-trunk.pdfei=prHhT-SMMMa70QG_uJTwAwusg=AFQjCNEq2fuo4dQrvFQT0zw8v05zMdIFWwsig2=6eAgFutMS_VLrhpR4Lhy2w Thanks, Joseph (w) 703-248-6160 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: Trinath Somanchi trinath.soman...@gmail.com To: openstack@lists.launchpad.net Sent: Wednesday, June 20, 2012 7:04:21 AM Subject: [Openstack] Testing NOVA-OVS-Quantum setup Hi- I have installed configured NOVA-OVS-Quantum based setup using the guide provided by openstack and OVS. I have a instance up and running. I'm new to Openstack. Can you any one help me out on the testing/validating on is instance up with OVS and Quantum. Thanking you.. -- Regards, -- Trinath Somanchi, +91 9866 235 130 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Filter Scheduler, a complete example
Christian, I have been working on improving filter scheduler and my current blue print is at http://wiki.openstack.org/InstanceTypeExtraSpecsExtension and current code is at https://review.openstack.org/#/c/8089/. If you have any question on my code or filter scheduler, please let me know. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: Christian Parpart tra...@gmail.com To: openstack@lists.launchpad.net Sent: Monday, June 11, 2012 11:41:42 AM Subject: [Openstack] Filter Scheduler, a complete example Hi all, while I am still somewhat new to OpenStack, I was able to successfully deploy a 6-node OpenStack setup with 4 compute nodes, one controller node (yet to be HA'd) and one network node (yet to be HA'd). However, now, that I am that far, I am in need to create a custom filter, due to the companies requirements on what VMs to put on what hardware. Unfortunately, I am not yet that experienced in Python (know quite a few others, so I shouldn't have it that hard in getting into) and I - of course - don't know the Nova API as well as you do. So I am looking for a complete basic Hello World Filter Scheduler example, e.g. in form of a github repo, that I can fork off, and improve it, and learn Nova by doing. I could not really find anything that helpful yet, and I think it might be a wonderful entry-point for quite a few of us. Is there anyone willing to help us here out a bit? Many many thanks, Christian Parpart. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Comparing roles - case (in)sensitivity
I'd vote case-sensitive. Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: Brian Waldon brian.wal...@rackspace.com To: openstack@lists.launchpad.net (openstack@lists.launchpad.net) openstack@lists.launchpad.net Sent: Friday, June 8, 2012 5:50:45 PM Subject: [Openstack] Comparing roles - case (in)sensitivity tl;dr - Should we compare roles as case-sensitive or case-insensitive? I vote case-sensitive. This bug was recently filed in Glance: https://bugs.launchpad.net/glance/+bug/1010519 . It points out that Nova and Keystone are both case-insensitive when it comes to role comparison, yet Glance *is* case sensitive. I'm in favor of moving other projects to a case-sensitive approach for two main reasons: 1) If a role is a string, and comparing strings is inherently case-sensitive, then role comparison would logically be case-sensitive 2) I get to do less work Thoughts? Brian Waldon ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Where to add the performance increasing method in openstack
Sarath, There are filter and weight functions in the scheduler that might fit your purpose. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: Razique Mahroua razique.mahr...@gmail.com To: sarath zacharia sarathzacha...@gmail.com Cc: openstack@lists.launchpad.net Sent: Wednesday, May 23, 2012 8:21:07 AM Subject: Re: [Openstack] Where to add the performance increasing method in openstack Hi, could we have extra details ? nova-scheduler is the service code you'd hack into :) sarath zacharia 23 mai 2012 07:26 Hi , We created an performance increasing programe for cloud application. This programe will check the performance and load of nodes according to the number of different application running in each node of our cloud environment and it will give which node is more suitable for deploying the next application in cloud. In which service we have to modify in openstack for implementing this algoritham ? note : now our code is in java how we can connect to the nova development environment Yours Sincerly Sarath Zacharia ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Nuage Co - Razique Mahroua razique.mahr...@gmail.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Community metrics, developers' engagement
I'd like to see how many companies are in the community in the statistics as well. Thanks, Joseph - Original Message - From: Jay Pipes jaypi...@gmail.com To: Anne Gentle a...@openstack.org Cc: openstack@lists.launchpad.net Sent: Tuesday, September 27, 2011 11:56:13 AM Subject: Re: [Openstack] Community metrics, developers' engagement I think it would awesome to have a stats.openstack.org (or similar) place to go where the community can view trends and graphs of statistics like what you mention below. Automation, FTW :) -jay On Tue, Sep 27, 2011 at 9:58 AM, Anne Gentle a...@openstack.org wrote: We could also add documentation statistics as they can tell a story about the supportive nature of the community. Would this be of interest to you? Here are some examples: April 22, 2011: 5 contributors to openstack-manuals 1,600 PDF downloads on docs.openstack.org 24,000 visits to docs.openstack.org 52,000 page views on docs.openstack.org Sept 26, 2011: 18 doc contributors to openstack-manuals 160,000 visits to docs.openstack.org (May 1 to Sept 22) 673,000 pageviews (May 1 to Sept 22) 9,000 PDF downloads (May 1 to Sept 22) These numbers are a good example showing the importance of tracking over time with such a young project and community. Also interesting to me are the search statistics and web statistics on the docs website. Would you all like regular reports of these? Let me know if it makes sense to include doc stats. Anne Anne Gentle | http://justwriteclick.com/ FacebookLinkedinTwitter Web Bug from http://p1.wisestamp.com/pixel.png?p=mozillav=2.8.5t=1317131535572u=755481b20c4ca423 On Mon, Sep 26, 2011 at 1:33 PM, Jay Pipes jaypi...@gmail.com wrote: On Mon, Sep 26, 2011 at 1:51 PM, John Dickinson m...@not.mn wrote: On Sep 26, 2011, at 12:10 PM, Mike Scherbakov wrote: Do you mean something different from the summaries like this one? http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-09-20-22.15.html Yeah, it's not clear what was actually discussed. I'm looking for some summary. I'm not sure if it worth effort to do summary from day to day conversations on #openstack... seems to be too much work. I too would like to see official community and PPB meeting summaries. Ideally, I think it'd be nice for the summaries to be written by someone not involved in the discussion[1], but that wouldn't be required. Something between what the meetbot produces and the raw IRC logs would be nice. ++ -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] IPMI plugin for Scheduler
Kyle, Please wait until sprint planning meeting since we don't know if this user story will make it in this sprint. We can start working if the user story is included in this (or future) sprint task list. Thanks, Joseph - Original Message - From: Kyle Dunn kd...@isi.edu To: openstack@lists.launchpad.net Sent: Monday, June 6, 2011 7:45:51 AM Subject: [Openstack] IPMI plugin for Scheduler I am planning to develop an IPMI power control plugin for Nova Scheduler and would like to solicit feature requests and implementation suggestions from the list. This is still in early planning stage so any and all feedback is welcome. Thanks, Kyle Dunn USC-ISI Intern ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How to limit the total virtual processors/memory for one compute node?
Huang, If you are willing to modify code, you might want to take a look at the code at lp:~usc-isi/nova/hpc-trunk that has a scheduler (nova/scheduler/arch.py) that does not allow creating new instances if cpu or other resources are used up. If you have any question on the branch, please feel free to ask me. Thanks, Joseph - Original Message - From: Lorin Hochstein lo...@isi.edu To: Huang Zhiteng winsto...@gmail.com Cc: openstack@lists.launchpad.net Sent: Monday, May 23, 2011 11:00:22 PM Subject: Re: [Openstack] How to limit the total virtual processors/memory for one compute node? Hi Huang: You can use the simple scheduler, allocates new instances to hosts that have the fewest instances currently running. --scheduler_driver=nova.scheduler.simple.SimpleScheduler More sophisticated schedulers are currently under active development, but they haven't made it to the trunk yet. Take care, Lorin -- Lorin Hochstein, Computer Scientist USC Information Sciences Institute 703.812.3710 http://www.east.isi.edu/~lorin On May 23, 2011, at 10:00 PM, Huang Zhiteng wrote: Hi all, In my setup of Cactus, I found Nova scheduler would place newly created instance to a compute node that is already full occupied (in terms of memory or # of virtual processors), which lead to swapping and VP overcommitting. That would cause serious performance issue on a busy environment. So I was wondering if there's some kinda mechanism to limit to resource one compute node could use, something like the 'weight' in OpenNebula. I'm using Cactus (with GridDynamic's RHEL package), default scheduler policy, one zone only. Any suggestion? -- Regards Huang Zhiteng ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Extending Openstack's scheduling capabilities
Luis, You can also take a look at USC-ISI's scheduler for heterogeneous systems that checks resource availability as well. Currently, it checks number of cpus, memory space, and hard disk space. If available resource is not enough, it does not allow a new instance. The branch can be found at lp:~usc-isi/nova/hpc-trunk. Thanks, Joseph - Original Message - From: Luis Miguel Silva luismiguelferreirasi...@gmail.com To: openstack@lists.launchpad.net Sent: Thursday, May 5, 2011 9:08:02 PM Subject: [Openstack] Extending Openstack's scheduling capabilities Dear all, I'm evaluating Openstack and i'm interested in extending it's scheduling capabilities. So i'm looking for ways to replace Openstack's scheduler component with my own (so i can route the requests to an external scheduler that the company i work for develops). Any ideas on how i can do it? During my research, i found that there are 3 scheduling types in Openstack: - chance scheduler - zone scheduler - and the simple scheduler Is the scheduler component in Openstack modular / easy to replace? What options do i have? p.s. i've also found a post saying (http://webcache.googleusercontent.com/search?q=cache:AI0tmD2qMxsJ:https://bugs.launchpad.net/nova/%2Bbug/766444+openstack+bypass+schedulercd=1hl=enct=clnkgl=ussource=www.google.com): I've seen plenty of questions come in asking how to launch a VM on a particular compute node. I learned this is available with the burnin merge, where you can bypass the scheduler, but no documentation was included with that code. Please document it - I can review and edit as needed. Any idea where i can get more information on the subject? Thanks in advance, Luis Silva ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
