Re: [Openstack] [ceilometer] Monitoring physical devices
On Tue, Nov 06 2012, Graf Lucas (graflu0) wrote: I'm a little confused now... ;) Is the bare metal run by the platform operator the physical machine? What do you mean with the bare metal run to replace virtual instances for any project? AFAIU, bare-metal provisionning is about using hardware (bare-metal) rather than virtual instances as a flavor in Nova. For such case, we won't be able to poll anything about the hardware. For hardware ran by the operator, this will be doable. -- Julien Danjou /* Free Software hacker freelance http://julien.danjou.info */ pgpDsJxrCp10y.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Tue, Nov 6, 2012 at 10:53 PM, Julien Danjou jul...@danjou.info wrote: On Tue, Nov 06 2012, Graf Lucas (graflu0) wrote: I'm a little confused now... ;) Is the bare metal run by the platform operator the physical machine? What do you mean with the bare metal run to replace virtual instances for any project? AFAIU, bare-metal provisionning is about using hardware (bare-metal) rather than virtual instances as a flavor in Nova. For such case, we won't be able to poll anything about the hardware. For hardware ran by the operator, this will be doable. We can still talk to the IPMI controller for the machine, which will get us lots of info, without running agents in the host os. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Cloud Services ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
There is a use case for base metal hardware metering in the private cloud where the user allocated the machine does not have root access to kill the metering. Being able to create a single metering infrastructure for the entire private cloud, virtual or bare-metal allocation, is a need technically, it is not clear how to guarantee it but it is worth exploring. Tim -Original Message- From: openstack-bounces+tim.bell=cern...@lists.launchpad.net [mailto:openstack-bounces+tim.bell=cern...@lists.launchpad.net] On Behalf Of Robert Collins Sent: 06 November 2012 11:00 To: Graf Lucas (graflu0); Zehnder Toni (zehndton); openstack@lists.launchpad.net; Doug Hellmann Subject: Re: [Openstack] [ceilometer] Monitoring physical devices On Tue, Nov 6, 2012 at 10:53 PM, Julien Danjou jul...@danjou.info wrote: On Tue, Nov 06 2012, Graf Lucas (graflu0) wrote: I'm a little confused now... ;) Is the bare metal run by the platform operator the physical machine? What do you mean with the bare metal run to replace virtual instances for any project? AFAIU, bare-metal provisionning is about using hardware (bare-metal) rather than virtual instances as a flavor in Nova. For such case, we won't be able to poll anything about the hardware. For hardware ran by the operator, this will be doable. We can still talk to the IPMI controller for the machine, which will get us lots of info, without running agents in the host os. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Cloud Services ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp smime.p7s Description: S/MIME cryptographic signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Nov 6, 2012, at 6:45 AM, Tim Bell tim.b...@cern.ch wrote: There is a use case for base metal hardware metering in the private cloud where the user allocated the machine does not have root access to kill the metering. How can we detect that special case? Being able to create a single metering infrastructure for the entire private cloud, virtual or bare-metal allocation, is a need technically, it is not clear how to guarantee it but it is worth exploring. I agree, it would be good to have an answer. Ceilometer can already hold the data, even if the agent to collect it is a custom solution. Doug Tim -Original Message- From: openstack-bounces+tim.bell=cern...@lists.launchpad.net [mailto:openstack-bounces+tim.bell=cern...@lists.launchpad.net] On Behalf Of Robert Collins Sent: 06 November 2012 11:00 To: Graf Lucas (graflu0); Zehnder Toni (zehndton); openstack@lists.launchpad.net; Doug Hellmann Subject: Re: [Openstack] [ceilometer] Monitoring physical devices On Tue, Nov 6, 2012 at 10:53 PM, Julien Danjou jul...@danjou.info wrote: On Tue, Nov 06 2012, Graf Lucas (graflu0) wrote: I'm a little confused now... ;) Is the bare metal run by the platform operator the physical machine? What do you mean with the bare metal run to replace virtual instances for any project? AFAIU, bare-metal provisionning is about using hardware (bare-metal) rather than virtual instances as a flavor in Nova. For such case, we won't be able to poll anything about the hardware. For hardware ran by the operator, this will be doable. We can still talk to the IPMI controller for the machine, which will get us lots of info, without running agents in the host os. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Cloud Services ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Thu, Nov 01 2012, Julien Danjou wrote: On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: My goal is to offer monitored data to the admin and customers. The admin is interested in the utilization of the physical components and the virtual machines and the customer is interested to know what his VMs do or can do. It would be nice to get the data from a single point. I thought I can enhance the Ceilometer compute agent to get this data out. Does this make sense or is it better to use another monitoring tool for the physical components? I think the pollster implementation can be done. I wouldn't implement this in the compute agent, but probably in some hardware agent, because it's likely it would be used in different kinds of environment and not only on compute node, i.e. you may also want to meter hardware usage for you cinder or glance node anyway. I think also the best way to implement this is to integrate a new (hardware) agent. Then we have a clear delineation. I'm very interested in helping to develop this. Toni About the 10 minutes polling interval Doug mentionned, this can be a problem indeed, but it's still solvable later and would be easy to solve if this in a different agent, since you could change the periodic interval for pollster runs to something like 1 or 5 minutes. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Fri, Nov 2, 2012 at 3:07 AM, Patrick Petit patrick.michel.pe...@gmail.com wrote: Folks, I'd like to add to this that physical server metering shouldn't be treated differently in Ceilometer now that bare metal provisioning framework enters into Grizzly. Physical servers will just become billable resources much like VMs. I am not speaking of physical server monitoring here. Just extending Ceilometer agent to also report usage data out of the physical box. Thanks for posting this, Patrick. I understood physical devices as host server not as bare-metal server. I suspect we could use the same agent framework, but almost all of the pollsters would need to be different because they would be running inside the guest OS rather than on a host VM, so the APIs they will use to collect the same data will be different. Doug Cheers Patrick Envoyé de mon iPad Le 1 nov. 2012 à 19:13, Julien Danjou jul...@danjou.info a écrit : On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: My goal is to offer monitored data to the admin and customers. The admin is interested in the utilization of the physical components and the virtual machines and the customer is interested to know what his VMs do or can do. It would be nice to get the data from a single point. I thought I can enhance the Ceilometer compute agent to get this data out. Does this make sense or is it better to use another monitoring tool for the physical components? I think the pollster implementation can be done. I wouldn't implement this in the compute agent, but probably in some hardware agent, because it's likely it would be used in different kinds of environment and not only on compute node, i.e. you may also want to meter hardware usage for you cinder or glance node anyway. About the 10 minutes polling interval Doug mentionned, this can be a problem indeed, but it's still solvable later and would be easy to solve if this in a different agent, since you could change the periodic interval for pollster runs to something like 1 or 5 minutes. -- Julien Danjou // Free Software hacker freelance // http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Mon, Nov 05 2012, Doug Hellmann wrote: If we make the current compute agent take an option telling it which pollster namespace to use, then the same framework can load different pollsters. However, there is a fundamental security issue with communicating from an agent running inside a tenant's OS image using the RPC stack. At DreamHost, and I suspect at other providers, that RPC network is completely isolated from any tenant networks. We would not want a tenant to be able to listen to the message bus, and definitely would not want it to be able to write anything to the message bus. What makes you think an agent would run inside an instance? I mean, this is not what this is about, we're talking about hardware running OS. -- Julien Danjou # Free Software hacker freelance # http://julien.danjou.info pgp7Pk0gZAZEV.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Mon, Nov 05 2012, Doug Hellmann wrote: When an image is deployed to bare metal, there is no container, right? Ah, I see the confusion. There's 2 bare metal, I think, the ones run by the the platform operator and the ones run to replace virtual instances for any project. I was actually talking about the former in this thread so far. :) For the latter, there's indeed this kind of problem, but I don't think we really want to meter resources on that. Well, at least I don't see the point and how it can be safe anyway. -- Julien Danjou # Free Software hacker freelance # http://julien.danjou.info pgpnuRPSMxHAW.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
Folks, I'd like to add to this that physical server metering shouldn't be treated differently in Ceilometer now that bare metal provisioning framework enters into Grizzly. Physical servers will just become billable resources much like VMs. I am not speaking of physical server monitoring here. Just extending Ceilometer agent to also report usage data out of the physical box. Cheers Patrick Envoyé de mon iPad Le 1 nov. 2012 à 19:13, Julien Danjou jul...@danjou.info a écrit : On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: My goal is to offer monitored data to the admin and customers. The admin is interested in the utilization of the physical components and the virtual machines and the customer is interested to know what his VMs do or can do. It would be nice to get the data from a single point. I thought I can enhance the Ceilometer compute agent to get this data out. Does this make sense or is it better to use another monitoring tool for the physical components? I think the pollster implementation can be done. I wouldn't implement this in the compute agent, but probably in some hardware agent, because it's likely it would be used in different kinds of environment and not only on compute node, i.e. you may also want to meter hardware usage for you cinder or glance node anyway. About the 10 minutes polling interval Doug mentionned, this can be a problem indeed, but it's still solvable later and would be easy to solve if this in a different agent, since you could change the periodic interval for pollster runs to something like 1 or 5 minutes. -- Julien Danjou // Free Software hacker freelance // http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Monitoring physical devices
Hi there, I am looking for a way to monitor the whole OpenStack environment including physical servers as well as the network. On every physical compute node is the Ceilometer compute agent installed, right?! 1) Does the compute agent collect data of the physical machine as well or is it just collecting data of the virtual machines? 2) Could it be useful to enhance the Ceilometer agent to collect data from the physical servers? Toni ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: On every physical compute node is the Ceilometer compute agent installed, right?! Yes. 1) Does the compute agent collect data of the physical machine as well or is it just collecting data of the virtual machines? Only virtual machines. 2) Could it be useful to enhance the Ceilometer agent to collect data from the physical servers? Why not. What do you have in mind exactly? -- Julien Danjou ;; Free Software hacker freelance ;; http://julien.danjou.info pgpOmfnhSmJUB.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: On Thu, Nov 01 2012, Julien Danjou wrote: On every physical compute node is the Ceilometer compute agent installed, right?! Yes. 1) Does the compute agent collect data of the physical machine as well or is it just collecting data of the virtual machines? Only virtual machines. 2) Could it be useful to enhance the Ceilometer agent to collect data from the physical servers? Why not. What do you have in mind exactly? My goal is to offer monitored data to the admin and customers. The admin is interested in the utilization of the physical components and the virtual machines and the customer is interested to know what his VMs do or can do. It would be nice to get the data from a single point. I thought I can enhance the Ceilometer compute agent to get this data out. Does this make sense or is it better to use another monitoring tool for the physical components? Cheers, Toni ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Thu, Nov 1, 2012 at 1:21 PM, Zehnder Toni (zehndton) zehnd...@students.zhaw.ch wrote: On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: On Thu, Nov 01 2012, Julien Danjou wrote: On every physical compute node is the Ceilometer compute agent installed, right?! Yes. 1) Does the compute agent collect data of the physical machine as well or is it just collecting data of the virtual machines? Only virtual machines. 2) Could it be useful to enhance the Ceilometer agent to collect data from the physical servers? Why not. What do you have in mind exactly? My goal is to offer monitored data to the admin and customers. The admin is interested in the utilization of the physical components and the virtual machines and the customer is interested to know what his VMs do or can do. It would be nice to get the data from a single point. I thought I can enhance the Ceilometer compute agent to get this data out. Does this make sense or is it better to use another monitoring tool for the physical components? We are looking into adding monitoring features to ceilometer, but it is likely that your admin will want fresher (and different) data than we are collecting right now (the agent only polls every 10 minutes at this point). If you have a strong need for this, it would be great if you could work with us to help get it implemented more quickly. Doug Cheers, Toni ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: My goal is to offer monitored data to the admin and customers. The admin is interested in the utilization of the physical components and the virtual machines and the customer is interested to know what his VMs do or can do. It would be nice to get the data from a single point. I thought I can enhance the Ceilometer compute agent to get this data out. Does this make sense or is it better to use another monitoring tool for the physical components? I think the pollster implementation can be done. I wouldn't implement this in the compute agent, but probably in some hardware agent, because it's likely it would be used in different kinds of environment and not only on compute node, i.e. you may also want to meter hardware usage for you cinder or glance node anyway. About the 10 minutes polling interval Doug mentionned, this can be a problem indeed, but it's still solvable later and would be easy to solve if this in a different agent, since you could change the periodic interval for pollster runs to something like 1 or 5 minutes. -- Julien Danjou // Free Software hacker freelance // http://julien.danjou.info pgpDAWYZn0Ulm.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Thu, Nov 1, 2012 at 2:13 PM, Julien Danjou jul...@danjou.info wrote: On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: My goal is to offer monitored data to the admin and customers. The admin is interested in the utilization of the physical components and the virtual machines and the customer is interested to know what his VMs do or can do. It would be nice to get the data from a single point. I thought I can enhance the Ceilometer compute agent to get this data out. Does this make sense or is it better to use another monitoring tool for the physical components? I think the pollster implementation can be done. I wouldn't implement this in the compute agent, but probably in some hardware agent, because it's likely it would be used in different kinds of environment and not only on compute node, i.e. you may also want to meter hardware usage for you cinder or glance node anyway. About the 10 minutes polling interval Doug mentionned, this can be a problem indeed, but it's still solvable later and would be easy to solve if this in a different agent, since you could change the periodic interval for pollster runs to something like 1 or 5 minutes. Good points. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
