Re: [Openstack] Potential filter scheduler enhancement
Phil, I agree with the suggestion. We wanted to instantiate a set of VMs whose hosts are located close each other. The current scheduler could not handle this use case since it needed to return "optimal" set of hosts at the same time. The first "optimal" host returned by the current scheduler may not be a part of the set of "optimal" hosts. We worked on the scheduler a little (in draft mode), and if you are interested in it, please let me know. Thanks, Joseph - Original Message - From: "Vishvananda Ishaya" To: "Phil Day" Cc: "openstack@lists.launchpad.net (openstack@lists.launchpad.net) (openstack@lists.launchpad.net)" Sent: Thursday, January 3, 2013 12:43:02 PM Subject: Re: [Openstack] Potential filter scheduler enhancement I think this seems reasonable, although FYI, openstack-dev seems like a better place for emails like this. Vish On Jan 3, 2013, at 6:40 AM, "Day, Phil" < philip@hp.com > wrote: Hi Folks, and Happy New Year. In working with the Filter Scheduler I’m considering an enhancement to make the final host selection stage configurable. Whilst its sometimes fine to just pick the first host from the list of weighted hosts, the more general case is that I’d like to be able to have the scheduler pick one of the first N hosts on the weighted list. The specific use cases that I have in mind are: - On a large system there is very rarely a single ideal / optimal host for a particular instance to be placed on. In practice any of the N most suitable hosts would be fine and allowing the scheduler to randomly pick one of these would add some spread for multiple requests that come in at the same time. (I know we now have the retry mechanism if a particular host can’t in fact handle a specific request – this is a complement to that rather an alternative). Of course anyone who wants to schedule to host in strict weighted order would be able to configure N to be 1 (or we could keep the current host selection method as a separate default) - When creating M instances in one request we could just put each onto one of the first M hosts in the list (since these have all been filtered as being suitable) instead of having to iterate through the filter / weighting functions for each successive instance. Based on this I’m thinking that such a host_selection function would replace the whole of the for loop at the end of the _schedule() method in filter_scheduler.py, and take as input the number of instances. The default function would of course be the current behaviour. Before going any further with this thinking I wanted to get input on: i) Do others recognise these use cases as being useful, and are there other similar use cases to be considered at the same time ? ii) Is it reasonable to make the filter scheduler configurable in this way, rather than creating a separate scheduler ? (My feeling is that because it would only be replacing ~10% of the current filter_scheduler code it would be better to not create a new scheduler) iii) Should the configuration logic for this new function be in the fliter_scheduler itself, or in the host_manager (which is where the filter and weighting functions are configured) ? Cheers, Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-hpc] [HPC] Reminder monthly telecon Sep. 10
All, I have a blue print on proximity scheduler at http://wiki.openstack.org/ProximityScheduler, and would like to get feedback on it. Thanks, Joseph - Original Message - From: "John Paul Walters" To: openstack@lists.launchpad.net Cc: openstack-...@lists.openstack.org Sent: Friday, September 7, 2012 12:12:20 PM Subject: [openstack-hpc] [HPC] Reminder monthly telecon Sep. 10 Hi, This is a reminder that we'll hold our next monthly HPC telecon this coming Monday, Sep. 10 and 12:00 noon Eastern Time. We'll use webex (details below). The agenda is somewhat open. Our default will be to start the conversation about HPC features that folks are interested in adding to the Grizzly release. If anyone has any other specific agenda items, they're welcome to propose them. I'm unable to attend, so my colleague David Kang will be hosting this meeting. We look forward to talking to you! best, JP John Paul Walters invites you to attend this online meeting. Topic: HPC Monthly Telecon Date: Monday, September 10, 2012 Time: 12:00 pm, Eastern Daylight Time (New York, GMT-04:00) Meeting Number: 927 246 497 Meeting Password: hpcmonthly --- To join the online meeting (Now from mobile devices!) --- 1. Go to https://openstack.webex.com/openstack/j.php?ED=203524102&UID=1431607857&PW=NYzljOTEwYThj&RT=MiMxMQ%3D%3D 2. If requested, enter your name and email address. 3. If a password is required, enter the meeting password: hpcmonthly 4. Click "Join". To view in other time zones or languages, please click the link: https://openstack.webex.com/openstack/j.php?ED=203524102&UID=1431607857&PW=NYzljOTEwYThj&ORT=MiMxMQ%3D%3D ___ OpenStack-HPC mailing list openstack-...@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-hpc ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Question regarding who the nova database is updated
Heng, I maybe wrong as the latest code is somewhat different from the one I followed last time, but it looks like the information is update at line 440 in nova/nova/compute/resource_tracker.py. Thanks, Joseph - Original Message - From: "Heng Xu" To: openstack@lists.launchpad.net Sent: Tuesday, August 28, 2012 1:52:18 PM Subject: [Openstack] Question regarding who the nova database is updated Hi folks: I was trying to add a new column to the Nova database table compute_nodes. however, I was not sure of how it is updated periodically, all I know is that comsume_from_instance() function in host_manager.py update the memory, then I was not able to find how nova subsequently update the compute_nodes table in the database. Any help will be appreciated, please let me know the exact location (file, function, etc.) and how I can mimc that to modify my own new field in the compute_nodes table. Thanks in advance. Heng ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
A bug fix patch that adds the extra specs matching against host aggregate is under review at https://review.openstack.org/#/c/11449/. Thanks, Joseph - Original Message - From: "Vishvananda Ishaya" To: "Joseph Suh" Cc: "Patrick Petit" , "openstack@lists.launchpad.net (openstack@lists.launchpad.net)" Sent: Friday, August 24, 2012 10:10:05 PM Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications Folsom also supports setting key values for things like capabilities via host aggregates. There is a filter[1] that matches the extra specs by exact comparison just like was done for capabilities before the last patch. The new extra specs matching should be added to it. These capabilities can be set dynamically by administrators so it directly supports the use case below. [1] https://github.com/openstack/nova/blob/master/nova/scheduler/filters/aggregate_instance_extra_specs.py On Aug 24, 2012, at 6:39 PM, Joseph Suh wrote: > Patrick, > > That's a good point. I think the issue is already being discussed at > https://bugs.launchpad.net/nova/+bug/1039386 as Don Dugger pointed out. > > That being said, as answers to some of your questions: yes, any key/value > pair can be used and it is user's (in this case, system admin's) > responsibility to avoid conflict at this time. The scope we originally > thought was pretty much static like the number of GPUs, but there is no > reason why it should be static as some features can change dynamically. I'd > encourage you to propose a blueprint if you can. We can also consider the > feature, but our team needs to discuss it before we can commit to it. > > Thanks, > > Joseph > > - Original Message - > From: "Patrick Petit" > To: "Joseph Suh" > Cc: "openstack@lists.launchpad.net (openstack@lists.launchpad.net)" > , "David Kang" > Sent: Friday, August 24, 2012 7:37:31 PM > Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications > > Hi Joseph and All, > > You are pointing the root cause of my question. The question is about how to > specify capabilities for a compute node so that they can be compared with the > extra specs. I think how to define extra specs in a flavor is clear enough. > > So, some capabilities are standard and are generated dynamically. Others are > not known or not captured by the system and so not standard (yet), like the > GPUs, and therefore must be specified somehow. Today, the somehow, as I > understand things, is to add key/value pairs in nova.conf when/if it is > supported. > > I wanted to make sure i understand the basic principals. > > Now, this in my opinion poses couple problems and/or call for additional > questions: > > What is the naming convention to add capabilities in nova.conf? I would > suppose that any key/value pair cannot be taken for a capability. > > How to avoid name clashing with standard capability? At the very least one > should have an option to print them out (in nova-manage?). Even a simple > written list would be helpful. > > But, are we really comfortable with the idea to define static capabilities in > nova.conf? that's putting a lot of burden on config management. Also, not > standard doesn't imply static. > > We can certainly live we that for now, But eventually, i think we'll need > some sort of an extension mechanism so that providers can generate whatever > capabilities they want using their own plugin? Note that capabilities could > be software related too. > > What do you think? > Best, > Patrick > > Envoyé de mon iPad > > Le 24 août 2012 à 18:38, Joseph Suh a écrit : > >> Patrick, >> >> Once a new item (key and value pair) is added to the capabilities, it can be >> compared against extra_specs. The extra_specs can be populated in >> instance_type_extra_specs table. The items in the extra_specs can start with >> one of the keywords for operations such as ">=" and "s==". For example, if >> "ngpus: 4" is populated in capability, extra_specs of ">= 2" will choose the >> host. If the extra_specs is ">= 5", the host will not be chosen. If no >> keyword is found at the beginning of the extra_specs (with the latest change >> in upstream code), the given string is directly compared with capability. >> For example, if "fpu" is given as extra_specs, the capability must be "fpu" >> to be selected. >> >> If more clarification is needed, please let us know. >> >> Thanks, >> >> Joseph >> >> - Or
Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
Patrick, That's a good point. I think the issue is already being discussed at https://bugs.launchpad.net/nova/+bug/1039386 as Don Dugger pointed out. That being said, as answers to some of your questions: yes, any key/value pair can be used and it is user's (in this case, system admin's) responsibility to avoid conflict at this time. The scope we originally thought was pretty much static like the number of GPUs, but there is no reason why it should be static as some features can change dynamically. I'd encourage you to propose a blueprint if you can. We can also consider the feature, but our team needs to discuss it before we can commit to it. Thanks, Joseph - Original Message - From: "Patrick Petit" To: "Joseph Suh" Cc: "openstack@lists.launchpad.net (openstack@lists.launchpad.net)" , "David Kang" Sent: Friday, August 24, 2012 7:37:31 PM Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications Hi Joseph and All, You are pointing the root cause of my question. The question is about how to specify capabilities for a compute node so that they can be compared with the extra specs. I think how to define extra specs in a flavor is clear enough. So, some capabilities are standard and are generated dynamically. Others are not known or not captured by the system and so not standard (yet), like the GPUs, and therefore must be specified somehow. Today, the somehow, as I understand things, is to add key/value pairs in nova.conf when/if it is supported. I wanted to make sure i understand the basic principals. Now, this in my opinion poses couple problems and/or call for additional questions: What is the naming convention to add capabilities in nova.conf? I would suppose that any key/value pair cannot be taken for a capability. How to avoid name clashing with standard capability? At the very least one should have an option to print them out (in nova-manage?). Even a simple written list would be helpful. But, are we really comfortable with the idea to define static capabilities in nova.conf? that's putting a lot of burden on config management. Also, not standard doesn't imply static. We can certainly live we that for now, But eventually, i think we'll need some sort of an extension mechanism so that providers can generate whatever capabilities they want using their own plugin? Note that capabilities could be software related too. What do you think? Best, Patrick Envoyé de mon iPad Le 24 août 2012 à 18:38, Joseph Suh a écrit : > Patrick, > > Once a new item (key and value pair) is added to the capabilities, it can be > compared against extra_specs. The extra_specs can be populated in > instance_type_extra_specs table. The items in the extra_specs can start with > one of the keywords for operations such as ">=" and "s==". For example, if > "ngpus: 4" is populated in capability, extra_specs of ">= 2" will choose the > host. If the extra_specs is ">= 5", the host will not be chosen. If no > keyword is found at the beginning of the extra_specs (with the latest change > in upstream code), the given string is directly compared with capability. For > example, if "fpu" is given as extra_specs, the capability must be "fpu" to be > selected. > > If more clarification is needed, please let us know. > > Thanks, > > Joseph > > - Original Message - > From: "David Kang" > To: "Patrick Petit" > Cc: "openstack@lists.launchpad.net (openstack@lists.launchpad.net)" > > Sent: Friday, August 24, 2012 11:34:11 AM > Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications > > > Parick, > > We are using the feature in Bare-metal machine provisioning. > Some keys are automatically generated by nova-compute. > For example, "hypervisor_type", "hypervisor_version", etc. fields are > automatically > put into capabilities by nova-compute (in the case of libvirt). > So, you don't need to specify that. > But, if you want to add custom fields, you should put them into nova.conf > file of > the nova-compute node. > > Since the new key are put into 'capabilities', > the new key must be different from any other keys in the 'capabilities'. > If that uniqueness is enforced, it can be any string, I believe. > > Thanks, > David > > -- > Dr. Dong-In "David" Kang > Computer Scientist > USC/ISI > > - Original Message - >> Hi, >> >> >> Could someone give a practical overview of how configuring and using >> the instance type extra specs extension capability introduced in >> Folsom? >
Re: [Openstack] [Nova] Instance Type Extra Specs clarifications
Patrick, Once a new item (key and value pair) is added to the capabilities, it can be compared against extra_specs. The extra_specs can be populated in instance_type_extra_specs table. The items in the extra_specs can start with one of the keywords for operations such as ">=" and "s==". For example, if "ngpus: 4" is populated in capability, extra_specs of ">= 2" will choose the host. If the extra_specs is ">= 5", the host will not be chosen. If no keyword is found at the beginning of the extra_specs (with the latest change in upstream code), the given string is directly compared with capability. For example, if "fpu" is given as extra_specs, the capability must be "fpu" to be selected. If more clarification is needed, please let us know. Thanks, Joseph - Original Message - From: "David Kang" To: "Patrick Petit" Cc: "openstack@lists.launchpad.net (openstack@lists.launchpad.net)" Sent: Friday, August 24, 2012 11:34:11 AM Subject: Re: [Openstack] [Nova] Instance Type Extra Specs clarifications Parick, We are using the feature in Bare-metal machine provisioning. Some keys are automatically generated by nova-compute. For example, "hypervisor_type", "hypervisor_version", etc. fields are automatically put into capabilities by nova-compute (in the case of libvirt). So, you don't need to specify that. But, if you want to add custom fields, you should put them into nova.conf file of the nova-compute node. Since the new key are put into 'capabilities', the new key must be different from any other keys in the 'capabilities'. If that uniqueness is enforced, it can be any string, I believe. Thanks, David -- Dr. Dong-In "David" Kang Computer Scientist USC/ISI - Original Message - > Hi, > > > Could someone give a practical overview of how configuring and using > the instance type extra specs extension capability introduced in > Folsom? > > > If how extending an instance type is relatively clear. > > > Eg.: #nova-manage instance_type set_key --name= --key > --value <'s== x86_64'> > > > The principles of capability advertising is less clearer. Is it > assumed that the key/value pairs are always declared statically as > flags in nova.conf of the compute node, or can they be generated > dynamically and if so, who would that be? And also, are the keys > completely free form strings or strings that are known (reserved) by > Nova? > > > Thanks in advance for clarifying this. > > > Patrick > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Unable to start nova-scheduler : duplicate option: scheduler_host_manager
Ben, It is possible to have the option specified in a code. Try grep -r on whole code. Thanks, Joseph - Original Message - From: "Ben" To: "Joseph Suh" Cc: openstack@lists.launchpad.net Sent: Wednesday, August 22, 2012 10:47:35 AM Subject: Re: [Openstack] Unable to start nova-scheduler : duplicate option: scheduler_host_manager Hi Joseph, Thank you for your answer. Yes, but where could be that option ? It's not duplicated in my nova.conf file, and this is the file the nova-scheduler uses. My nova.conf file joined. Regards, Ben Le 22/08/2012 16:37, Joseph Suh a écrit : > Ben, > > As the error message suggests, it is due to a duplicated option of > scheduler_host_manager. It is specified more than once somewhere. > > Thanks, > > Joseph > > - Original Message - > From: "Ben" > To: openstack@lists.launchpad.net > Sent: Wednesday, August 22, 2012 8:27:39 AM > Subject: Re: [Openstack] Unable to start nova-scheduler : duplicate option: > scheduler_host_manager > > Hi, > > No idea for my scheduler problem ? It was working at the beginning (I > suppose because I could launch instances), but I did a lot of > modification, modifying networks, deleting nova db, rebuild it, etc. > > Any idea on what I can do to identify the problem ? Where can I find the > mentionned option except in nova.conf ? > > Ben > > Le 22/08/2012 00:46, Ben a écrit : >> Hi, >> >> I'm trying to setup a little nova cluster with 3 nodes : >> >> - 1 controller node running all services but compute >> - 2 compute nodes running compute and network only >> >> I have faced a lot of issues, but I can't understand this one. When I >> start nova-scheduler on controller node, the process dies instantly with >> this error : >> >> CRITICAL nova [-] duplicate option: scheduler_host_manager >> >> So I can't start an instance, it remains stuck in building state. I have >> checked my nova.conf file, and I only see this line : >> >> --scheduler_driver=nova.scheduler.simple.SimpleScheduler >> >> What does means this error, how can I solve it ? >> >> Thanks, >> >> Ben > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Unable to start nova-scheduler : duplicate option: scheduler_host_manager
Ben, As the error message suggests, it is due to a duplicated option of scheduler_host_manager. It is specified more than once somewhere. Thanks, Joseph - Original Message - From: "Ben" To: openstack@lists.launchpad.net Sent: Wednesday, August 22, 2012 8:27:39 AM Subject: Re: [Openstack] Unable to start nova-scheduler : duplicate option: scheduler_host_manager Hi, No idea for my scheduler problem ? It was working at the beginning (I suppose because I could launch instances), but I did a lot of modification, modifying networks, deleting nova db, rebuild it, etc. Any idea on what I can do to identify the problem ? Where can I find the mentionned option except in nova.conf ? Ben Le 22/08/2012 00:46, Ben a écrit : > Hi, > > I'm trying to setup a little nova cluster with 3 nodes : > > - 1 controller node running all services but compute > - 2 compute nodes running compute and network only > > I have faced a lot of issues, but I can't understand this one. When I > start nova-scheduler on controller node, the process dies instantly with > this error : > > CRITICAL nova [-] duplicate option: scheduler_host_manager > > So I can't start an instance, it remains stuck in building state. I have > checked my nova.conf file, and I only see this line : > > --scheduler_driver=nova.scheduler.simple.SimpleScheduler > > What does means this error, how can I solve it ? > > Thanks, > > Ben ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using extra specs
Boris-Michel, One thing that I noticed was a typo: schedulre that can cause malfunction. I am not sure what version you are using, but recently the extra_spec checking is moved to compute_capabilities_filter.py (ComputeCapabilitiesFilter). As far as I understand, the current ComputeFilter does not check extra_specs. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: "Boris-Michel Deschenes" To: openstack@lists.launchpad.net Sent: Thursday, August 9, 2012 11:10:15 AM Subject: [Openstack] using extra specs Hi guys, I currently have a working cloud with a working GPU passthrough setup (CentOS/libvirt/Xen 4.1.2), now I need to work on adding this new resource to openstack. Here is the plan: 1. Create a new instance type (g1.small) with an extra spec like “xpu_arch = radeon” 2. Modify the compute side so that nova-compute publishes this new capability I’m in between step 1 and step 2, so normally, when I try and launch a VM with the new instance type, the scheduling should fail since there is no compute node publishing this capability yet, but the scheduling does go through despite having the ComputeFilter available. My understanding is that by having: scheduler_available_filters=nova.schedulre.filters.standard_filters scheduler_default_filters=ComputeFilter The ComputeFilter should try and ensure that the hosts satisfies the extra specs but the filter is not doing its job since the VM is scheduler and casted to the compute node although it does not advertise any of the extra spec. The extra spec is definitely part of the instance_type, I can see it in the scheduler log: (TRUNCATED) u'8895af5063b176bef97ab81f076d57f3', u'min_disk': u'0', u'is_public': True, u'deleted_at': None, u'properties': {u'os_type': u'windows'}, u'size': 9256562688}, u'instance_type': {u'root_gb': 0, u'deleted_at': None, u'name': u'g1.small', u'deleted': False, u'created_at': u'2012-08-08 18:33:29', u'ephemeral_gb': 0, u'updated_at': None, u'memory_mb': 2048, u'vcpus': 1, u'extra_specs': {u'xpu_arch': u'radeon'}, u'swap': 0, u'rxtx_factor': 1.0, u'flavorid': u'105', u'vcpu_weight': None, u'id': 6}, u'instance_properties': {u'vm_state': u'building', u'availability_zone': (TRUNCATED) So again, this VM should not be casted to any compute node and the scheduler should complain about no host having the required capability but it does not… the VM is instantiated on the one compute node even though it never published the required capability. It looks like the ComputeFilter is either not used properly or I’m missing something. Basically I want the scheduler to fail when I launch the VM before I start working on step 2. Any help appreciated. Boris ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cannot pass hint to Nova Scheduler
Heng, You can print the values in the HostState class. If you want to monitor the changes of the values, for example, you can print (either directly or using LOG.debug()), the values in the code where you want monitor, for example, consume_from_instance(). Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: "Jay Pipes" To: "Heng Xu" Cc: openstack@lists.launchpad.net Sent: Monday, August 6, 2012 12:11:58 PM Subject: Re: [Openstack] Cannot pass hint to Nova Scheduler On 08/04/2012 07:48 AM, Heng Xu wrote: > But I tried a few things in HostState class, I ran into error, because I > could not monitor the stats in hoststates class as opposed to a database, is > there a way to check the stats in HostState class as exists in memory? cc'ing Sandy Walsh, who is vastly more familiar with the scheduler than I am :) Sandy, see Heng's question above... seems like a great question to me -- and also a possible mini-project for someone to work on that would add a scheduler-diagnostics extension if such functionality isn't readily available. Best, -jay > Heng > > From: Jay Pipes [jaypi...@gmail.com] > Sent: Friday, August 03, 2012 4:38 PM > To: Heng Xu > Cc: openstack@lists.launchpad.net > Subject: Re: [Openstack] Cannot pass hint to Nova Scheduler > > On 08/03/2012 09:28 AM, Heng Xu wrote: >> Another questions is, I can get all the status of a computing node in the >> mysql nova database, and select * from compute_node, but now I am using json >> filter, the only field I have success with now is the free_ram_db, if my >> hint uses free_disk_gb, then I always get error, but the database is showing >> my compute node has $free_disk_gb equal 17, so I was wondering, where to >> find exactly what kind of json field can use in json filter, thanks in >> advance > > The nova.scheduler.host_manager.HostState class is what is checked for > attributes, not the ComputeNode model. So, you need to use > "$free_disk_mb", not "free_disk_gb". > > Best, > -jay > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cannot pass hint to Nova Scheduler
Heng, Does scheduler log show any error message or complaints? Thanks, Joseph (w) 703-248-6160 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: "Heng Xu" To: openstack@lists.launchpad.net Sent: Thursday, August 2, 2012 10:57:53 AM Subject: [Openstack] Cannot pass hint to Nova Scheduler Hi folks: I am new to openstack, I am current trying to test the json filter, I changed my /etc/nova/nova.conf as follow scheduler_driver=nova. scheduler.multi.MultiScheduler compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler volume_scheduler_driver=nova.scheduler.chance.ChanceScheduler scheduler_available_filters=nova.scheduler.filters.standard_filters scheduler_default_filters=JsonFilter least_cost_functions=nova.scheduler.least_cost.compute_fill_first_cost_fn compute_fill_first_cost_fn_weight=-1.0 so I can use the json filter however, when I was using it, if I boot a vm without any --hint to the scheduler, then the vm started fine, but if I use nova --debug boot --image 827d564a-e636-4fc4-a376- d36f7ebe1747 --flavor 1 --hint query=['>=','$free_ram_mb',1024] server1 my vm started with error, and the following were output from the command above +- +--+ | Property | Value | +-+--+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-SRV-ATTR:host | None | | OS-EXT-SRV-ATTR:hypervisor_hostname | None | | OS-EXT-SRV-ATTR:instance_name | instance-002b | | OS-EXT-STS:power_state | 0 | | OS-EXT-STS:task_state | scheduling | | OS-EXT-STS:vm_state | error | | accessIPv4 | | | accessIPv6 | | | adminPass | dKvrsv4MZtfc | | config_drive | | | created | 2012-08-02T14:25:10Z | | flavor | m1.tiny | | hostId | | | id | 9d4a5855-3c69-40ba-b50d-3a2aa6a92edc | | image | cirros-0.3.0-x86_64-uec | | key_name | | | metadata | {} | | name | server1 | | progress | 0 | | status | BUILD | | tenant_id | d99ffa1b0c43455ab8dbbd81cf4380a7 | | updated | 2012-08-02T14:25:10Z | | user_id | d5e02f1810a44575b99a147f94507da1 | +-+--+ as you can see, the vm is in error, this also happens whenever I need to pass a hint to the scheduler, as in samehostfilter and differenthostfilter, Does anyone know what's going on, thanks in advance. Heng ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] One question on the compute_filter
Don, Hmm, so the use of the instance_type_extra_specs is not for specifications purely any more... I am open to your proposed idea that if there is no operator, it will ignore the item (that is not original behavior of compute_filter, though). So, let me wait and see if there are any objections/comments on this issue from community, and if I don't hear them, I will implement my patch in that way. Thanks, Joseph (w) 703-248-6160 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: "Donald D Dugger" To: "Joseph Suh" , "Sandy Walsh" , "Vishvananda Ishaya" Cc: openstack@lists.launchpad.net, "Yunhong Jiang" Sent: Tuesday, July 17, 2012 12:19:35 PM Subject: RE: One question on the compute_filter Joseph- Basically, Vish & Sandy convinced me that the `instance_type_extra_specs' was the right place to put the trust info, rather than creating a new table. I like the idea of expanding the idea of extra_specs to be used to store extra information. I can easily imagine cases where someone wants to create a new, specialized filter that requires more information (the trusted filter is one example, there could be others). Especially if we want the ability to add filters that aren't part of the `official` source tree, and therefore can’t change standard APIs, you'll need a place to store more info and the `instance_type_extra_specs' table is the obvious place. Adding in comparison operators, especially the equality operator, would make it easy to have all the filters co-exist. -- Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale Ph: 303/443-3786 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] One question on the compute_filter
Don, That's an interesting idea, but I am having a difficulty in understanding why you want to store extra information (not spec) in instance_type_extra_specs. If you need to keep some extra information, using or creating another flag or field in database might be a better place for that? The intention of having instance_type_extra_specs is to extend the instance_types table to accommodate extra specs that cannot be enumerated at the time of coding, and it would be best if we have the same behavior as instance_types in my opinion. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: "Donald D Dugger" To: "Joseph Suh" , "Yunhong Jiang" Cc: openstack@lists.launchpad.net Sent: Tuesday, July 17, 2012 11:47:25 AM Subject: RE: One question on the compute_filter Joeseph- My concern is that this means that the compute filter consumes `every` entry in the `extra_specs' table. I can imagine scenarios where other filters would want to put data in `extra_specs' that is not intended for the compute filter. In fact, we do that today with the trusted filter that was recently added where there is a `trusted_host' key that is added to the `extra_specs' table. I really like the idea of adding operators to the compute filter. This would mean that if you truly wanted the compute filter to check for the presence of capability you just explicitly specify the equality operator. Any `extra_specs' keys that don't have an operator would just be ignored, giving the administrator maximum flexibility in specifying capabilities. -- Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale Ph: 303/443-3786 -Original Message- From: Joseph Suh [mailto:j...@isi.edu] Sent: Tuesday, July 17, 2012 7:38 AM To: Jiang, Yunhong Cc: Dugger, Donald D; openstack@lists.launchpad.net Subject: Re: One question on the compute_filter Yunhong, Thanks for your interest in our patch. The original purpose of the instance_type_extra_specs is providing ability to specify any specs that cannot be enumerated at the nova coding time such as hardware-specific requirements (like gpu) or any features in the future system (like python version 10). System administrator will populate necessary items at the time of deployment as he/she wants. Current compute_filter is a relatively simple code that compares the items in the instance_type_extra_specs with those in capability. In our patch, we want to add more operators, not changing the basic behavior. The idea behind the compute_filter is that if items are specified in instance_type_extra_specs, the filter makes it sure that all the requested items are in the capability of the provisioned hosts. So, in your approach, if you already know items that is required such as xpu_arch and want to ignore other items in the instance_type_extra_spec, one way might be to create another filter that checks only those known items. But, then, if there is a need to check another item or ignore an item in the future, the filter code needs to be modified. In our approach, it can be simply added/deleted in database by system administrator. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: "Yunhong Jiang" To: j...@isi.edu Cc: "Donald D Dugger" , openstack@lists.launchpad.net Sent: Monday, July 16, 2012 10:00:44 AM Subject: One question on the compute_filter Hi, Joseph I’m working on the patch for blueprints https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value , to add/delete the extra_specs for flavor through nova-manage. I’m still setting up my environment to push the patch. However, when I was testing my patch, I noticed that compute_filter assume it will handle all of the “extra_specs”. If it can’t find corresponding key in the capabilities, it will fail to pass the host. IMHO, this is a bit overkill. For example, currently the trusted_filter.py will use the extra_specs to check if trusted_host is required, that means compute filter and trusted filter can’t be used at the same time. I think compute filter should define explicitly all keys that it takes care, like cpu_info, cpu_arch, xpu_arch, and only check the corresponding extra_specs key/value pair? After all, extra_specs is not compute_extra_specs? I noticed the patch in https://review.openstack.org/#/c/8089/ , but seems this patch will not fix this issue still. Any idea or suggestion? I’m glad to create patch if any conclusion on this issue. Thanks --jyh ___
Re: [Openstack] One question on the compute_filter
Yunhong, I understand your concern. It has a different purpose than the compute_filter, so it has its own merit and can co-exist. The question is how much the demand is... Thanks, Joseph (w) 703-248-6160 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: "Yunhong Jiang" To: "Joseph Suh" Cc: "Donald D Dugger" , openstack@lists.launchpad.net Sent: Tuesday, July 17, 2012 11:39:54 AM Subject: RE: One question on the compute_filter > -----Original Message- > From: Joseph Suh [mailto:j...@isi.edu] > Sent: Tuesday, July 17, 2012 9:38 PM > To: Jiang, Yunhong > Cc: Dugger, Donald D; openstack@lists.launchpad.net > Subject: Re: One question on the compute_filter > > Yunhong, > > Thanks for your interest in our patch. The original purpose of the > instance_type_extra_specs is providing ability to specify any specs that > cannot > be enumerated at the nova coding time such as hardware-specific > requirements (like gpu) or any features in the future system (like python > version > 10). System administrator will populate necessary items at the time of > deployment as he/she wants. Current compute_filter is a relatively simple code > that compares the items in the instance_type_extra_specs with those in > capability. In our patch, we want to add more operators, not changing the > basic > behavior. Joseph, thanks for clarification. > > The idea behind the compute_filter is that if items are specified in > instance_type_extra_specs, the filter makes it sure that all the requested > items are in the capability of the provisioned hosts. > > So, in your approach, if you already know items that is required such as > xpu_arch and want to ignore other items in the instance_type_extra_spec, one > way might be to create another filter that checks only those known items. But, The issue here is, if I create such another filter, then it possibly can't co-exist with compute filter :( --jyh > then, if there is a need to check another item or ignore an item in the > future, > the filter code needs to be modified. In our approach, it can be simply > added/deleted in database by system administrator. > > Thanks, > > Joseph > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] One question on the compute_filter
Yunhong, Thanks for your interest in our patch. The original purpose of the instance_type_extra_specs is providing ability to specify any specs that cannot be enumerated at the nova coding time such as hardware-specific requirements (like gpu) or any features in the future system (like python version 10). System administrator will populate necessary items at the time of deployment as he/she wants. Current compute_filter is a relatively simple code that compares the items in the instance_type_extra_specs with those in capability. In our patch, we want to add more operators, not changing the basic behavior. The idea behind the compute_filter is that if items are specified in instance_type_extra_specs, the filter makes it sure that all the requested items are in the capability of the provisioned hosts. So, in your approach, if you already know items that is required such as xpu_arch and want to ignore other items in the instance_type_extra_spec, one way might be to create another filter that checks only those known items. But, then, if there is a need to check another item or ignore an item in the future, the filter code needs to be modified. In our approach, it can be simply added/deleted in database by system administrator. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: "Yunhong Jiang" To: j...@isi.edu Cc: "Donald D Dugger" , openstack@lists.launchpad.net Sent: Monday, July 16, 2012 10:00:44 AM Subject: One question on the compute_filter Hi, Joseph I’m working on the patch for blueprints https://blueprints.launchpad.net/nova/+spec/update-flavor-key-value , to add/delete the extra_specs for flavor through nova-manage. I’m still setting up my environment to push the patch. However, when I was testing my patch, I noticed that compute_filter assume it will handle all of the “extra_specs”. If it can’t find corresponding key in the capabilities, it will fail to pass the host. IMHO, this is a bit overkill. For example, currently the trusted_filter.py will use the extra_specs to check if trusted_host is required, that means compute filter and trusted filter can’t be used at the same time. I think compute filter should define explicitly all keys that it takes care, like cpu_info, cpu_arch, xpu_arch, and only check the corresponding extra_specs key/value pair? After all, extra_specs is not compute_extra_specs? I noticed the patch in https://review.openstack.org/#/c/8089/ , but seems this patch will not fix this issue still. Any idea or suggestion? I’m glad to create patch if any conclusion on this issue. Thanks --jyh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack] duplicate option: scheduler_host_manager
Leander, As the error message indicates, it usually comes when the option is found in more than one place. Thanks, Joseph (w) 703-248-6160 (f) 703-812-3712 http://www.east.isi.edu/~jsuh Information Sciences Institute University of Southern California 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA - Original Message - From: "Leander Bessa Beernaert" To: openstack@lists.launchpad.net Sent: Friday, June 29, 2012 11:45:49 AM Subject: [Openstack] [OpenStack] duplicate option: scheduler_host_manager Hello, I'm working on this patch https://review.openstack.org/#/c/8839/ and i keep running into this error https://jenkins.openstack.org/job/gate-nova-python26/1333/testReport/%3Cnose.suite/ContextSuite%20context=nova/tests__setup/ Can someone shed some light on the subject? Regards, Leander ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Jenkins
Gary, Apparently, it happened to me. Tests took 13 hours and came with failure. Thanks, Joseph (w) 703-248-6160 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: "Gary Kotton" To: openstack@lists.launchpad.net Sent: Wednesday, June 27, 2012 1:04:28 PM Subject: [Openstack] Jenkins Hi, Is anyone aware of a problem with Jenkins? Thanks Gary ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] No tests available in custom branch
Leander, If it works fine in master branch, the problem must be due to some changes in your own branch. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: "Leander Bessa Beernaert" To: "Joseph Suh" Cc: openstack@lists.launchpad.net Sent: Monday, June 25, 2012 9:45:51 AM Subject: Re: [Openstack] [OpenStack][Nova] No tests available in custom branch I've tried all the options and the output is always, even with -V and -f: -- Ran 0 tests in 0.000s OK Running PEP8 and HACKING compliance check... 5 imports missing in this test environment The tests run just fine on the master branch :/ On Mon, Jun 25, 2012 at 2:39 PM, Joseph Suh < j...@isi.edu > wrote: Leander, Did you give -V -f option? If you post your log, it would be easier to help. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: "Leander Bessa Beernaert" < leande...@gmail.com > To: openstack@lists.launchpad.net Sent: Monday, June 25, 2012 9:13:43 AM Subject: [Openstack] [OpenStack][Nova] No tests available in custom branch Hello, I've created a separate branch in Nova to test my code out. However, every time i attempt to run the command "run_tests.sh" it says it has 0 tests to be run. I followed the guide from http://wiki.openstack.org/GerritWorkflow to setup the branch. Did i miss something? Regards, Leander ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] No tests available in custom branch
Leander, Did you give -V -f option? If you post your log, it would be easier to help. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: "Leander Bessa Beernaert" To: openstack@lists.launchpad.net Sent: Monday, June 25, 2012 9:13:43 AM Subject: [Openstack] [OpenStack][Nova] No tests available in custom branch Hello, I've created a separate branch in Nova to test my code out. However, every time i attempt to run the command "run_tests.sh" it says it has 0 tests to be run. I followed the guide from http://wiki.openstack.org/GerritWorkflow to setup the branch. Did i miss something? Regards, Leander ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Testing NOVA-OVS-Quantum setup
Trinath, I found the following Quantun admin guide was useful for that purpose: http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=4&ved=0CHsQFjAD&url=http%3A%2F%2Fdocs.openstack.org%2Ftrunk%2Fopenstack-network%2Fadmin%2Fquantum-admin-guide-trunk.pdf&ei=prHhT-SMMMa70QG_uJTwAw&usg=AFQjCNEq2fuo4dQrvFQT0zw8v05zMdIFWw&sig2=6eAgFutMS_VLrhpR4Lhy2w Thanks, Joseph (w) 703-248-6160 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: "Trinath Somanchi" To: openstack@lists.launchpad.net Sent: Wednesday, June 20, 2012 7:04:21 AM Subject: [Openstack] Testing NOVA-OVS-Quantum setup Hi- I have installed configured NOVA-OVS-Quantum based setup using the guide provided by openstack and OVS. I have a instance up and running. I'm new to Openstack. Can you any one help me out on the testing/validating on is instance up with OVS and Quantum. Thanking you.. -- Regards, -- Trinath Somanchi, +91 9866 235 130 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Filter Scheduler, a complete example
Christian, I have been working on improving filter scheduler and my current blue print is at http://wiki.openstack.org/InstanceTypeExtraSpecsExtension and current code is at https://review.openstack.org/#/c/8089/. If you have any question on my code or filter scheduler, please let me know. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: "Christian Parpart" To: openstack@lists.launchpad.net Sent: Monday, June 11, 2012 11:41:42 AM Subject: [Openstack] Filter Scheduler, a complete example Hi all, while I am still "somewhat" new to OpenStack, I was able to successfully deploy a 6-node OpenStack setup with 4 compute nodes, one controller node (yet to be HA'd) and one network node (yet to be HA'd). However, now, that I am that far, I am in need to create a custom filter, due to the companies requirements on what VMs to put on what hardware. Unfortunately, I am not yet that experienced in Python (know quite a few others, so I shouldn't have it that hard in getting into) and I - of course - don't know the Nova API as well as you do. So I am looking for a complete basic Hello World Filter Scheduler example, e.g. in form of a github repo, that I can fork off, and improve it, and learn Nova by doing. I could not really find anything that helpful yet, and I think it might be a wonderful entry-point for quite a few of us. Is there anyone willing to help us here out a bit? Many many thanks, Christian Parpart. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Comparing roles - case (in)sensitivity
I'd vote case-sensitive. Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: "Brian Waldon" To: "openstack@lists.launchpad.net (openstack@lists.launchpad.net)" Sent: Friday, June 8, 2012 5:50:45 PM Subject: [Openstack] Comparing roles - case (in)sensitivity tl;dr - Should we compare roles as case-sensitive or case-insensitive? I vote case-sensitive. This bug was recently filed in Glance: https://bugs.launchpad.net/glance/+bug/1010519 . It points out that Nova and Keystone are both case-insensitive when it comes to role comparison, yet Glance *is* case sensitive. I'm in favor of moving other projects to a case-sensitive approach for two main reasons: 1) If a role is a string, and comparing strings is inherently case-sensitive, then role comparison would logically be case-sensitive 2) I get to do less work Thoughts? Brian Waldon ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Where to add the performance increasing method in openstack
Sarath, There are filter and weight functions in the scheduler that might fit your purpose. Thanks, Joseph (w) 703-248-6160 (c) 571-340-2434 (f) 703-812-3712 3811 N. Fairfax Drive Suite 200 Arlington, VA, 22203, USA http://www.east.isi.edu/~jsuh - Original Message - From: "Razique Mahroua" To: "sarath zacharia" Cc: openstack@lists.launchpad.net Sent: Wednesday, May 23, 2012 8:21:07 AM Subject: Re: [Openstack] Where to add the performance increasing method in openstack Hi, could we have extra details ? nova-scheduler is the service code you'd hack into :) sarath zacharia 23 mai 2012 07:26 Hi , We created an performance increasing programe for cloud application. This programe will check the performance and load of nodes according to the number of different application running in each node of our cloud environment and it will give which node is more suitable for deploying the next application in cloud. In which service we have to modify in openstack for implementing this algoritham ? note : now our code is in java how we can connect to the nova development environment Yours Sincerly Sarath Zacharia ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Nuage & Co - Razique Mahroua razique.mahr...@gmail.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Community metrics, developers' engagement
I'd like to see how many companies are in the community in the statistics as well. Thanks, Joseph - Original Message - From: "Jay Pipes" To: "Anne Gentle" Cc: openstack@lists.launchpad.net Sent: Tuesday, September 27, 2011 11:56:13 AM Subject: Re: [Openstack] Community metrics, developers' engagement I think it would awesome to have a stats.openstack.org (or similar) place to go where the community can view trends and graphs of statistics like what you mention below. Automation, FTW :) -jay On Tue, Sep 27, 2011 at 9:58 AM, Anne Gentle < a...@openstack.org > wrote: We could also add documentation statistics as they can tell a story about the supportive nature of the community. Would this be of interest to you? Here are some examples: April 22, 2011: 5 contributors to openstack-manuals 1,600 PDF downloads on docs.openstack.org 24,000 visits to docs.openstack.org 52,000 page views on docs.openstack.org Sept 26, 2011: 18 doc contributors to openstack-manuals 160,000 visits to docs.openstack.org (May 1 to Sept 22) 673,000 pageviews (May 1 to Sept 22) 9,000 PDF downloads (May 1 to Sept 22) These numbers are a good example showing the importance of tracking over time with such a young project and community. Also interesting to me are the search statistics and web statistics on the docs website. Would you all like regular reports of these? Let me know if it makes sense to include doc stats. Anne Anne Gentle | http://justwriteclick.com/ FacebookLinkedinTwitter Web Bug from http://p1.wisestamp.com/pixel.png?p=mozilla&v=2.8.5&t=1317131535572&u=755481b20c4ca423 On Mon, Sep 26, 2011 at 1:33 PM, Jay Pipes < jaypi...@gmail.com > wrote: On Mon, Sep 26, 2011 at 1:51 PM, John Dickinson < m...@not.mn > wrote: > On Sep 26, 2011, at 12:10 PM, Mike Scherbakov wrote: >> >> Do you mean something different from the summaries like this one? >> >> http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-09-20-22.15.html >> >> >> Yeah, it's not clear what was actually discussed. I'm looking for some >> summary. >> I'm not sure if it worth effort to do summary from day to day conversations >> on #openstack... >> seems to be too much work. > > I too would like to see official community and PPB meeting summaries. > Ideally, I think it'd be nice for the summaries to be written by someone not > involved in the discussion[1], but that wouldn't be required. Something > between what the meetbot produces and the raw IRC logs would be nice. ++ -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] IPMI plugin for Scheduler
Kyle, Please wait until sprint planning meeting since we don't know if this user story will make it in this sprint. We can start working if the user story is included in this (or future) sprint task list. Thanks, Joseph - Original Message - From: "Kyle Dunn" To: openstack@lists.launchpad.net Sent: Monday, June 6, 2011 7:45:51 AM Subject: [Openstack] IPMI plugin for Scheduler I am planning to develop an IPMI power control plugin for Nova Scheduler and would like to solicit feature requests and implementation suggestions from the list. This is still in early planning stage so any and all feedback is welcome. Thanks, Kyle Dunn USC-ISI Intern ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How to limit the total virtual processors/memory for one compute node?
Huang, If you are willing to modify code, you might want to take a look at the code at lp:~usc-isi/nova/hpc-trunk that has a scheduler (nova/scheduler/arch.py) that does not allow creating new instances if cpu or other resources are used up. If you have any question on the branch, please feel free to ask me. Thanks, Joseph - Original Message - From: "Lorin Hochstein" To: "Huang Zhiteng" Cc: openstack@lists.launchpad.net Sent: Monday, May 23, 2011 11:00:22 PM Subject: Re: [Openstack] How to limit the total virtual processors/memory for one compute node? Hi Huang: You can use the simple scheduler, allocates new instances to hosts that have the fewest instances currently running. --scheduler_driver=nova.scheduler.simple.SimpleScheduler More sophisticated schedulers are currently under active development, but they haven't made it to the trunk yet. Take care, Lorin -- Lorin Hochstein, Computer Scientist USC Information Sciences Institute 703.812.3710 http://www.east.isi.edu/~lorin On May 23, 2011, at 10:00 PM, Huang Zhiteng wrote: Hi all, In my setup of Cactus, I found Nova scheduler would place newly created instance to a compute node that is already full occupied (in terms of memory or # of virtual processors), which lead to swapping and VP overcommitting. That would cause serious performance issue on a busy environment. So I was wondering if there's some kinda mechanism to limit to resource one compute node could use, something like the 'weight' in OpenNebula. I'm using Cactus (with GridDynamic's RHEL package), default scheduler policy, one zone only. Any suggestion? -- Regards Huang Zhiteng ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Extending Openstack's scheduling capabilities
Luis, You can also take a look at USC-ISI's scheduler for heterogeneous systems that checks resource availability as well. Currently, it checks number of cpus, memory space, and hard disk space. If available resource is not enough, it does not allow a new instance. The branch can be found at lp:~usc-isi/nova/hpc-trunk. Thanks, Joseph - Original Message - From: "Luis Miguel Silva" To: openstack@lists.launchpad.net Sent: Thursday, May 5, 2011 9:08:02 PM Subject: [Openstack] Extending Openstack's scheduling capabilities Dear all, I'm evaluating Openstack and i'm interested in extending it's scheduling capabilities. So i'm looking for ways to replace Openstack's scheduler component with my own (so i can route the requests to an external scheduler that the company i work for develops). Any ideas on how i can do it? During my research, i found that there are 3 scheduling types in Openstack: - chance scheduler - zone scheduler - and the simple scheduler Is the scheduler component in Openstack modular / easy to replace? What options do i have? p.s. i've also found a post saying (http://webcache.googleusercontent.com/search?q=cache:AI0tmD2qMxsJ:https://bugs.launchpad.net/nova/%2Bbug/766444+openstack+bypass+scheduler&cd=1&hl=en&ct=clnk&gl=us&source=www.google.com): "I've seen plenty of questions come in asking how to launch a VM on a particular compute node. I learned this is available with the burnin merge, where you can bypass the scheduler, but no documentation was included with that code. Please document it - I can review and edit as needed." Any idea where i can get more information on the subject? Thanks in advance, Luis Silva ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] distributed and heterogeneous schedulers
I think 1) is a clean solution. Another option is adding "frequently" used columns to the instance table (and/or compute nodes table), but this leaves a subjective decision on what is a frequently used column and needs constant change whenever a new decision is made. It seems that Jay's suggestion of adding is_user_added column is reasonable and a quick solution. Thanks, Joseph - Original Message - From: "Jay Pipes" To: "Mark Washenberger" Cc: openstack@lists.launchpad.net Sent: Thursday, April 14, 2011 11:29:27 AM Subject: Re: [Openstack] distributed and heterogeneous schedulers This is an excellent point, Mark, I'd forgotten about the quota system for user-defined (and nova-ignored) "metadata" items. That does throw a bit of a wrench into the mix. Perhaps a small edit to the instance_metadata table to add a column for is_user_added that can be used to segregate metadata items that Nova itself uses to track certain things? -jay On Thu, Apr 14, 2011 at 10:01 AM, Mark Washenberger wrote: >> 1) Continue to add fields to the instances table (or compute_nodes >> table) for these main attributes like cpu_arch, etc. >> 2) Use the custom key/value table (instance_metadata) to store these >> attribute names and their values >> 3) Do both 1) and 2) > > I've no particular preference here, but if we start adding attributes to > instance_metadata that nova code specifically knows about, we need to ensure > there is a good way to tell the difference between "nova" metadata and "user" > metadata. I do not believe key prefixes will provide this difference cleanly. > > Currently, instance_metadata is for user-provided attributes only and nova is > agnostic to their contents. In the api, they are presented as such, and there > are quotas on the number of metadata items the user can set. This would all > break if we start adding "nova" metadata without further api and db (small) > changes. > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp