[openstack-dev] [Nova] Power/temperature monitors for scheduler
Hi, all, I would like to propose adding power/temperature monitors in Nova for the scheduler. I builded a wiki to record the design notes, https://wiki.openstack.org/wiki/User_talk:Fengqian#Why_and_how_to_use_Intel_Node_Manager_in_Openstack Please help to review the following patches if you are interested. Add node manager to read power/temperature of compute node https://review.openstack.org/#/c/65631/ Added temperature monitor for compute node https://review.openstack.org/#/c/64403/ Added power monitor for compute node https://review.openstack.org/#/c/64404/ Many thanks for considering my request. Thanks --fengqian ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Power/temperature monitors for scheduler
For the ceilometer part, it seems have overlap, but I am not sure about Nova part. IIUC, Nova doesn't collect power/temperature for now. Thanks --fengqian -Original Message- From: Tim Bell [mailto:tim.b...@cern.ch] Sent: Thursday, January 23, 2014 2:21 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova] Power/temperature monitors for scheduler Is there overlap with https://launchpad.net/kwapi and https://blueprints.launchpad.net/ironic/+spec/send-data-to-ceilometer ? Tim From: Gao, Fengqian [mailto:fengqian@intel.com] Sent: 23 January 2014 06:25 To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [Nova] Power/temperature monitors for scheduler Hi, all, I would like to propose adding power/temperature monitors in Nova for the scheduler. I builded a wiki to record the design notes, https://wiki.openstack.org/wiki/User_talk:Fengqian#Why_and_how_to_use_Intel_Node_Manager_in_Openstack Please help to review the following patches if you are interested. Add node manager to read power/temperature of compute node https://review.openstack.org/#/c/65631/ Added temperature monitor for compute node https://review.openstack.org/#/c/64403/ Added power monitor for compute node https://review.openstack.org/#/c/64404/ Many thanks for considering my request. Thanks --fengqian ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ ceilometer] The Pagination patches
Hi, Jay, Thanks for your reply. According to you review comments, I rebase my code, please see my comments for your questions. For the issue that ensure there is at least one unique column in the sort keys if pagination is used, I have an idea. How about add the primary key at the end of sort keys list passed from users? Thanks --fengqian -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Monday, January 13, 2014 5:10 AM To: Gao, Fengqian Cc: Julien Danjou (jul...@danjou.info); Doug Hellmann (doug.hellm...@dreamhost.com); Lu, Lianhao; chu...@ca.ibm.com; dper...@linux.vnet.ibm.com; akornie...@mirantis.com; openstack-dev@lists.openstack.org Subject: Re: [openstack-dev][ ceilometer] The Pagination patches On Fri, 2014-01-10 at 03:31 +, Gao, Fengqian wrote: Since we still have no new conclusion for pagination for now, shall we still go on with the current pagination solution? Please help to review patches: https://review.openstack.org/#/c/41869/ https://review.openstack.org/#/c/35454/ Hi Fengqian, Please see my review on the first of the above patches. You need to ensure that there is at least one unique column in the sort keys if pagination is used. There is a similar issue in the second patch (reviewing it now) that uses the sqlalchemy_utils (oslo.db) paginate_query() method... The sqlalchemy_utils.paginate_query() method **requires** that sort_keys contains a unique column [1]. The docstring for that method says: Pagination works by requiring a unique sort_key, specified by sort_keys. (If sort_keys is not unique, then we risk looping through values.) So, care must be taken to pass sort_keys parameter to paginate_query() that already contains at least one unique column in the sort keys. Best, -jay [1] https://github.com/openstack/oslo-incubator/blob/master/openstack/common/db/sqlalchemy/utils.py#L66 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI
Hi, Pradipta, From personal experience, I think lm-sensors is not good as IPMI. I have to configure it manually and the sensor data it could get also less than IPMI. So, I prefer to use IPMI. Did you use it before? Maybe you can share your experience. Best wishes --fengqian From: Pradipta Banerjee [mailto:bprad...@yahoo.com] Sent: Friday, December 20, 2013 10:52 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI On 12/19/2013 12:30 AM, Devananda van der Veen wrote: On Tue, Dec 17, 2013 at 10:00 PM, Gao, Fengqian fengqian@intel.commailto:fengqian@intel.com wrote: Hi, all, I am planning to extend bp https://blueprints.launchpad.net/nova/+spec/utilization-aware-scheduling with power and temperature. In other words, power and temperature can be collected and used for nova-scheduler just as CPU utilization. This is a good idea and have definite use cases where one might want to optimize provisioning based on power consumption I have a question here. As you know, IPMI is used to get power and temperature and baremetal implements IPMI functions in Nova. But baremetal driver is being split out of nova, so if I want to change something to the IPMI, which part should I choose now? Nova or Ironic? Hi! A few thoughts... Firstly, new features should be geared towards Ironic, not the nova baremetal driver as it will be deprecated soon (https://blueprints.launchpad.net/nova/+spec/deprecate-baremetal-driver). That being said, I actually don't think you want to use IPMI for what you're describing at all, but maybe I'm wrong. When scheduling VMs with Nova, in many cases there is already an agent running locally, eg. nova-compute, and this agent is already supplying information to the scheduler. I think this is where the facilities for gathering power/temperature/etc (eg, via lm-sensors) should be placed, and it can reported back to the scheduler along with other usage statistics. +1 Using lm-sensors or equivalent seems better. Have a look at the following blueprint https://blueprints.launchpad.net/nova/+spec/extensible-resource-tracking If you think there's a compelling reason to use Ironic for this instead of lm-sensors, please clarify. Cheers, Devananda ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Regards, Pradipta ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI
Hi, Devananda, Thanks for your reply. I totally agree with your thought, nova-compute could gathering such information locally. And I think IPMI, using in-band connection, might be more suitable to be used to collect these sensor data. At the beginning, I thought it would better to put all IPMI part together in Ironic while there is part of code over there. Now, it looks like that I thought too much:). In all, power and temperature could be collected by IPMI locally in nova-compute. Best wishes --fengqian From: Devananda van der Veen [mailto:devananda@gmail.com] Sent: Friday, December 20, 2013 3:42 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI On Wed, Dec 18, 2013 at 7:16 PM, Gao, Fengqian fengqian@intel.commailto:fengqian@intel.com wrote: Hi, Devananda, I agree with you that new features should be towards Ironic. As you asked why use Ironic instead of lm-sensors, actually I just want to use IPMI instead of lm-sensors. I think it is reasonable to put the IPMI part into Ironic and we already did:). To get the sensors' information, I think IPMI is much more powerful than lm-sensors. Firstly, IPMI is flexible. Generally speaking, it provides two kinds of connections, in-bind and out-of-band. Out-of-band connection allows us to get sensors' status even without OS and CPU. In-band connection is quite similar to lm-sensors, It needs the OS kernel to get sensor data. Secondly, IPMI can gather more sensors' information that lm-sensors and it is easy to use. From my own experience, using IPMI can get all the sensor information that lm-sensors could get, such as temperature/voltage/fan. Besides that, IPMI can get power data and some OEM specific sensor data. Thirdly, I think IPMI is a common spec for most of OEMs. And most of servers are integrated with IPMI interface. As you sais, nova-compute is already supplying information to the scheduler and power/temperature should be gathered locally. IPMI can be used locally, the in-band connection. And there is a lot of open source library, such as OpenIPMI, FreeIPMI, which provide the interfaces to OS, just like lm-sensors. So, I prefer to use IPMI than lm-sensors. Please leave your comments if you disagree:). I see nothing wrong with nova-compute gathering such information locally. Whether you use lm-sensors or in-band IPMI is an implementation detail of how nova-compute would gather the information. However, I don't see how this has anything to do with Ironic or the nova-baremetal driver. These would gather information remotely (using out-of-band IPMI) for hardware controlled and deployed by these services. In most cases, nova-compute is not deployed by nova-compute (exception: if you're running TripleO). Hope that helps, -Deva ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI
Yes, Alan, you got me. Providing power/temperature to scheduler, set threshold or different weight, then the scheduler can boot VM on the most suitable node. Thanks --fengqian From: Alan Kavanagh [mailto:alan.kavan...@ericsson.com] Sent: Friday, December 20, 2013 11:58 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI Cheers Gao It definitely makes sense to collect additional metrics such as power and temperature, and make that available for selective decisions you would want to take. However, I am just wondering if you could realistically feed those metrics as variables for scheduling, this is the main part I feel is questionable. I assume then you would use temperature || power etc to gauge if you want to schedule another VM on a given node when a given temperature threshold is reached. Is this the main case you are thinking of Gao? Alan From: Gao, Fengqian [mailto:fengqian@intel.com] Sent: December-18-13 10:23 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI Hi, Alan, I think, for nova-scheduler it is better if we gather more information. And In today's DC, power and temperature are very important facts to considering. CPU/Memory utilization is not enough to describe nodes' status. Power/inlet temperature should be noticed. Best Wishes --fengqian From: Alan Kavanagh [mailto:alan.kavan...@ericsson.com] Sent: Thursday, December 19, 2013 2:14 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI Hi Gao What is the reason why you see it would be important to have these two additional metrics power and temperature for Nova to base scheduling on? Alan From: Gao, Fengqian [mailto:fengqian@intel.com] Sent: December-18-13 1:00 AM To: openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI Hi, all, I am planning to extend bp https://blueprints.launchpad.net/nova/+spec/utilization-aware-scheduling with power and temperature. In other words, power and temperature can be collected and used for nova-scheduler just as CPU utilization. I have a question here. As you know, IPMI is used to get power and temperature and baremetal implements IPMI functions in Nova. But baremetal driver is being split out of nova, so if I want to change something to the IPMI, which part should I choose now? Nova or Ironic? Best wishes --fengqian ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI
Hi, Devananda, I agree with you that new features should be towards Ironic. As you asked why use Ironic instead of lm-sensors, actually I just want to use IPMI instead of lm-sensors. I think it is reasonable to put the IPMI part into Ironic and we already did:). To get the sensors' information, I think IPMI is much more powerful than lm-sensors. Firstly, IPMI is flexible. Generally speaking, it provides two kinds of connections, in-bind and out-of-band. Out-of-band connection allows us to get sensors' status even without OS and CPU. In-band connection is quite similar to lm-sensors, It needs the OS kernel to get sensor data. Secondly, IPMI can gather more sensors' information that lm-sensors and it is easy to use. From my own experience, using IPMI can get all the sensor information that lm-sensors could get, such as temperature/voltage/fan. Besides that, IPMI can get power data and some OEM specific sensor data. Thirdly, I think IPMI is a common spec for most of OEMs. And most of servers are integrated with IPMI interface. As you sais, nova-compute is already supplying information to the scheduler and power/temperature should be gathered locally. IPMI can be used locally, the in-band connection. And there is a lot of open source library, such as OpenIPMI, FreeIPMI, which provide the interfaces to OS, just like lm-sensors. So, I prefer to use IPMI than lm-sensors. Please leave your comments if you disagree:). Best wishes --fengqian From: Devananda van der Veen [mailto:devananda@gmail.com] Sent: Thursday, December 19, 2013 3:00 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI On Tue, Dec 17, 2013 at 10:00 PM, Gao, Fengqian fengqian@intel.commailto:fengqian@intel.com wrote: Hi, all, I am planning to extend bp https://blueprints.launchpad.net/nova/+spec/utilization-aware-scheduling with power and temperature. In other words, power and temperature can be collected and used for nova-scheduler just as CPU utilization. I have a question here. As you know, IPMI is used to get power and temperature and baremetal implements IPMI functions in Nova. But baremetal driver is being split out of nova, so if I want to change something to the IPMI, which part should I choose now? Nova or Ironic? Hi! A few thoughts... Firstly, new features should be geared towards Ironic, not the nova baremetal driver as it will be deprecated soon (https://blueprints.launchpad.net/nova/+spec/deprecate-baremetal-driver). That being said, I actually don't think you want to use IPMI for what you're describing at all, but maybe I'm wrong. When scheduling VMs with Nova, in many cases there is already an agent running locally, eg. nova-compute, and this agent is already supplying information to the scheduler. I think this is where the facilities for gathering power/temperature/etc (eg, via lm-sensors) should be placed, and it can reported back to the scheduler along with other usage statistics. If you think there's a compelling reason to use Ironic for this instead of lm-sensors, please clarify. Cheers, Devananda ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI
Hi, Alan, I think, for nova-scheduler it is better if we gather more information. And In today's DC, power and temperature are very important facts to considering. CPU/Memory utilization is not enough to describe nodes' status. Power/inlet temperature should be noticed. Best Wishes --fengqian From: Alan Kavanagh [mailto:alan.kavan...@ericsson.com] Sent: Thursday, December 19, 2013 2:14 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI Hi Gao What is the reason why you see it would be important to have these two additional metrics power and temperature for Nova to base scheduling on? Alan From: Gao, Fengqian [mailto:fengqian@intel.com] Sent: December-18-13 1:00 AM To: openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: [openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI Hi, all, I am planning to extend bp https://blueprints.launchpad.net/nova/+spec/utilization-aware-scheduling with power and temperature. In other words, power and temperature can be collected and used for nova-scheduler just as CPU utilization. I have a question here. As you know, IPMI is used to get power and temperature and baremetal implements IPMI functions in Nova. But baremetal driver is being split out of nova, so if I want to change something to the IPMI, which part should I choose now? Nova or Ironic? Best wishes --fengqian ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova] [Ironic] Get power and temperature via IPMI
Hi, all, I am planning to extend bp https://blueprints.launchpad.net/nova/+spec/utilization-aware-scheduling with power and temperature. In other words, power and temperature can be collected and used for nova-scheduler just as CPU utilization. I have a question here. As you know, IPMI is used to get power and temperature and baremetal implements IPMI functions in Nova. But baremetal driver is being split out of nova, so if I want to change something to the IPMI, which part should I choose now? Nova or Ironic? Best wishes --fengqian ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ironic] A question about getting IPMI data
Re-send again because been rejected by system. Best Regards, -fengqian From: Gao, Fengqian Sent: Tuesday, November 19, 2013 2:06 PM To: openstack-dev@lists.openstack.org Cc: 'Devananda van der Veen'; 'jbjoh...@us.ibm.com'; Lu, Lianhao; Wang, Shane Subject: [Ironic] A question about getting IPMI data Hi, all, As the summit session https://etherpad.openstack.org/p/icehouse-summit-ceilometer-hardware-sensors said, ironic will expose IPMI data to ceilometer, but I have a couple questions here. 1. What kind of IPMI data will be collected? I assume that power, temperature or other sensor data is needed, right? 2. How do we get all these data? IIUC, ironic did not involve too much about IPMI for now, only used ipmitool and pyghmi module. Using IPMItool to execute command and parsing the output is easy to understand. But seems not a good way for me. The pyghmi only supports out-of-band(net-payload) now. So, I am wondering if we can extend the function of pyghmi to provide more interfaces? Be specifically, I think pyghmi can provide in-band IPMI interface including IPMB or system interface etc. and allowed to get IPMI data in the OS. Is all this make sense to you? Thanks for your response Best Regards, -fengqian ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
