Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-28 Thread Devananda van der Veen
On Nov 25, 2013 7:13 PM, Doug Hellmann doug.hellm...@dreamhost.com
wrote:




 On Mon, Nov 25, 2013 at 3:56 PM, Devananda van der Veen 
devananda@gmail.com wrote:

 Hi!

 Very good questions. I think most of them are directed towards the
Ceilometer team, but I have answered a few bits inline.


 On Mon, Nov 25, 2013 at 7:24 AM, wanghaomeng wanghaom...@163.com wrote:


 Hello all:

 Basically, I understand the solution is - Our Ironic will implement an
IPMI driver


 We will need to add a new interface -- for example,
ironic.drivers.base.BaseDriver:sensor and the corresponding
ironic.drivers.base.SensorInterface class, then implement this interface as
ironic.drivers.modules.ipmitool:IPMISensor

 We also need to define the methods this interface supports and what the
return data type is for each method. I imagine it may be something like:
 - SensorInterface.list_available_sensors(node) returns a list of sensor
names for that node
 - SensorInterface.get_measurements(node, list_of_sensor_names) returns a
dict of dicts, eg, { 'sensor_1': {'key': 'value'}, 'sensor_2': ...}


 (extendable framework for more drivers) to collect hardware sensor
data(cpu temp, fan speed, volts, etc) via IPMI protocol from hardware
server node, and emit the AMQP message to Ceilometer Collector, Ceilometer
have the framework to handle the valid sample message and save to the
database for data retrieving by consumer.

 Now, how do you think if we should clearly define the interface  data
model specifications between Ironic and Ceilometer to enable IPMI data
collecting, then our two team can start the coding together?


 I think this is just a matter of understanding Ceilometer's API so that
Ironic can emit messages in the correct format. You've got many good
questions for the Ceilometer team on this below.



 And I still have some concern with our interface and data model as
below, the spec need to be discussed and finalized:

 1. What is the Ceilometer sample data mandatory attributes, such as
instance_id/tenant_id/user_id/resource_id, if they are not  optional, where
are these data populated, from Ironic or Ceilomter side?


   name/type/unit/volume/timestamp - basic sample property, can be
populated from Ironic side as data source
   user_id/project_id/resource_id - Ironic or Ceilometer populate these
fields??


 Ceilometer knows nothing about resources unless it is told, so all of the
required fields have to be provided by the sender.



   resource_metadata - this is for Ceilometer metadata query, Ironic
know nothing for such resource metadata I think


 The resource metadata depends on the resource type, but should be all of
the user-visible attributes for that object stored in the database at the
time the measurement is taken. For example, for instances we (try to) get
all of the instance attributes.


We could send all the node.properties,  Getting into node.driver_info would
expose passwords and such, so we shouldn't send that.


   source - can we hard-code as 'hardware' as a source identifier?


 No, the source is the source of the user and project ids, not the source
of the measurement (the data source is implied by the meter name). The
default source for user and project is openstack to differentiate from an
add-on layer like a PaaS where there are different user or project ids.





 Ironic can cache the user_id and project_id of the instance. These will
not be present for unprovisioned nodes.

 I'm not sure what resource_id is in this context, perhaps the nova
instance_uuid? If so, Ironic has that as well.


 Do end-users know about bare metal servers before they are provisioned as
instances? Can a regular user, for example, as for the list of available
servers or find details about one by name or id?



There is an API service which exposes information about unprovisioned
servers. At the moment, it is admin-only. If you think of an end-user as
someone using tuskar, they will likely want to know about unprovisioned
servers.




 2. Not sure if our Ceilometer only accept the signed-message, if it is
case, how Ironic get the message trust for Ceilometer, and send the valid
message which can be accepted by Ceilometer Collector?


 I'm not sure it's appropriate for ironic to be sending messages using
ceilometer's sample format. We receive data from the other projects using
the more generic notification system, and that seems like the right tool to
use here, too. Unless the other ceilometer devs disagree?




 3. What is the Ceilometer sample data structure, and what is the min
data item set for the IPMI message be emitted to Collector?
   name/type/unit/volume/timestamp/source - is this min data item set?

 3. If the detailed data model should be defined for our IPMI data now?,
what is our the first version scope, how many IPMI data type we should
support? Here is a IPMI data sample list, I think we can support these as a
min set.
   Temperature - System Temp/CPU Temp
   FAN Speed in rpm - FAN 1/2/3/4/A
   Volts - 

Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-28 Thread Doug Hellmann
On Thu, Nov 28, 2013 at 1:00 PM, Devananda van der Veen 
devananda@gmail.com wrote:


 On Nov 25, 2013 7:13 PM, Doug Hellmann doug.hellm...@dreamhost.com
 wrote:
 
 
 
 
  On Mon, Nov 25, 2013 at 3:56 PM, Devananda van der Veen 
 devananda@gmail.com wrote:
 
  Hi!
 
  Very good questions. I think most of them are directed towards the
 Ceilometer team, but I have answered a few bits inline.
 
 
  On Mon, Nov 25, 2013 at 7:24 AM, wanghaomeng wanghaom...@163.com
 wrote:
 
 
  Hello all:
 
  Basically, I understand the solution is - Our Ironic will implement an
 IPMI driver
 
 
  We will need to add a new interface -- for example,
 ironic.drivers.base.BaseDriver:sensor and the corresponding
 ironic.drivers.base.SensorInterface class, then implement this interface as
 ironic.drivers.modules.ipmitool:IPMISensor
 
  We also need to define the methods this interface supports and what the
 return data type is for each method. I imagine it may be something like:
  - SensorInterface.list_available_sensors(node) returns a list of sensor
 names for that node
  - SensorInterface.get_measurements(node, list_of_sensor_names) returns
 a dict of dicts, eg, { 'sensor_1': {'key': 'value'}, 'sensor_2': ...}
 
 
  (extendable framework for more drivers) to collect hardware sensor
 data(cpu temp, fan speed, volts, etc) via IPMI protocol from hardware
 server node, and emit the AMQP message to Ceilometer Collector, Ceilometer
 have the framework to handle the valid sample message and save to the
 database for data retrieving by consumer.
 
  Now, how do you think if we should clearly define the interface  data
 model specifications between Ironic and Ceilometer to enable IPMI data
 collecting, then our two team can start the coding together?
 
 
  I think this is just a matter of understanding Ceilometer's API so that
 Ironic can emit messages in the correct format. You've got many good
 questions for the Ceilometer team on this below.
 
 
 
  And I still have some concern with our interface and data model as
 below, the spec need to be discussed and finalized:
 
  1. What is the Ceilometer sample data mandatory attributes, such as
 instance_id/tenant_id/user_id/resource_id, if they are not  optional, where
 are these data populated, from Ironic or Ceilomter side?
 
 
name/type/unit/volume/timestamp - basic sample property, can be
 populated from Ironic side as data source
user_id/project_id/resource_id - Ironic or Ceilometer populate these
 fields??
 
 
  Ceilometer knows nothing about resources unless it is told, so all of
 the required fields have to be provided by the sender.
 
 
 
resource_metadata - this is for Ceilometer metadata query, Ironic
 know nothing for such resource metadata I think
 
 
  The resource metadata depends on the resource type, but should be all of
 the user-visible attributes for that object stored in the database at the
 time the measurement is taken. For example, for instances we (try to) get
 all of the instance attributes.
 

 We could send all the node.properties,  Getting into node.driver_info
 would expose passwords and such, so we shouldn't send that.


Agreed.



 
source - can we hard-code as 'hardware' as a source identifier?
 
 
  No, the source is the source of the user and project ids, not the source
 of the measurement (the data source is implied by the meter name). The
 default source for user and project is openstack to differentiate from an
 add-on layer like a PaaS where there are different user or project ids.
 
 
 
 
 
  Ironic can cache the user_id and project_id of the instance. These will
 not be present for unprovisioned nodes.
 
  I'm not sure what resource_id is in this context, perhaps the nova
 instance_uuid? If so, Ironic has that as well.
 
 
  Do end-users know about bare metal servers before they are provisioned
 as instances? Can a regular user, for example, as for the list of available
 servers or find details about one by name or id?
 
 

 There is an API service which exposes information about unprovisioned
 servers. At the moment, it is admin-only. If you think of an end-user as
 someone using tuskar, they will likely want to know about unprovisioned
 servers.

OK, then some form of auditing event (similar to the instance and network
exists events) might make sense. I think those are a lower priority than
the standard CRUD events, though.

 
 
 
  2. Not sure if our Ceilometer only accept the signed-message, if it is
 case, how Ironic get the message trust for Ceilometer, and send the valid
 message which can be accepted by Ceilometer Collector?
 
 
  I'm not sure it's appropriate for ironic to be sending messages using
 ceilometer's sample format. We receive data from the other projects using
 the more generic notification system, and that seems like the right tool to
 use here, too. Unless the other ceilometer devs disagree?
 
 
 
 
  3. What is the Ceilometer sample data structure, and what is the min
 data item set for the IPMI message 

Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-26 Thread Gordon Chung
 2. Not sure if our Ceilometer only accept the signed-message, if it is 
case, how Ironic get the message trust for Ceilometer, and send the valid 
message which can be accepted by Ceilometer Collector?

 I'm not sure it's appropriate for ironic to be sending messages using 
ceilometer's sample format. We receive data from the other projects using 
the more generic notification system, and that seems like the right tool 
to use here, too. Unless the other ceilometer devs disagree?

agreed, depending on your target milestone i'd suggest keeping an eye on 
this bp as well: 
https://blueprints.launchpad.net/oslo/+spec/notification-structured 

cheers,
gordon chung

openstack, ibm software standards
email: chungg [at] ca.ibm.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-25 Thread Devananda van der Veen
Hi!

Very good questions. I think most of them are directed towards the
Ceilometer team, but I have answered a few bits inline.


On Mon, Nov 25, 2013 at 7:24 AM, wanghaomeng wanghaom...@163.com wrote:


 Hello all:

 Basically, I understand the solution is - Our Ironic will implement an
 IPMI driver


We will need to add a new interface -- for example,
ironic.drivers.base.BaseDriver:sensor and the corresponding
ironic.drivers.base.SensorInterface class, then implement this interface as
ironic.drivers.modules.ipmitool:IPMISensor

We also need to define the methods this interface supports and what the
return data type is for each method. I imagine it may be something like:
- SensorInterface.list_available_sensors(node) returns a list of sensor
names for that node
- SensorInterface.get_measurements(node, list_of_sensor_names) returns a
dict of dicts, eg, { 'sensor_1': {'key': 'value'}, 'sensor_2': ...}


 (extendable framework for more drivers) to collect hardware sensor
 data(cpu temp, fan speed, volts, etc) via IPMI protocol from hardware
 server node, and emit the AMQP message to Ceilometer Collector,
 Ceilometer have the framework to handle the valid sample message and save
 to the database for data retrieving by consumer.

 Now, how do you think if we should clearly define the *interface  data
 model *specifications between Ironic and Ceilometer to enable IPMI data
 collecting, then our two team can start the coding together?


I think this is just a matter of understanding Ceilometer's API so that
Ironic can emit messages in the correct format. You've got many good
questions for the Ceilometer team on this below.



 And I still have some concern with our interface and data model as below,
 the spec need to be discussed and finalized:

 1. What is the Ceilometer sample data mandatory attributes, such as
 instance_id/tenant_id/user_id/resource_id, if they are not  optional, where
 are these data populated, from Ironic or Ceilomter side?

   *name/type/unit/volume/timestamp* - basic sample property, can be
 populated from Ironic side as data source
   *user_id/project_id/resource_id* - Ironic or Ceilometer populate these
 fields??
   *resource_metadata - this is for Ceilometer metadata query, Ironic know
 nothing for such resource metadata I think*
   *source *- can we hard-code as 'hardware' as a source identifier?


Ironic can cache the user_id and project_id of the instance. These will not
be present for unprovisioned nodes.

I'm not sure what resource_id is in this context, perhaps the nova
instance_uuid? If so, Ironic has that as well.


 2. Not sure if our Ceilometer only accept the *signed-message*, if it is
 case, how Ironic get the message trust for Ceilometer, and send the valid
 message which can be accepted by Ceilometer Collector?

 3. What is the Ceilometer sample data structure, and what is the min data
 item set for the IPMI message be emitted to Collector?
   *name/type/unit/volume/**timestamp/source - is this min data item set?*

 3. If the detailed data model should be defined for our IPMI data now?,
 what is our the first version scope, how many IPMI data type we should
 support? Here is a IPMI data sample list, I think we can support these as a
 min set.
   *Temperature - System Temp/CPU Temp*
 *  FAN Speed in rpm - FAN 1/2/3/4/A*
 *  Volts - Vcore/3.3VCC/12V/VDIMM/5VCC/-12V/VBAT/VSB/AVCC*


I think that's a good starting list. We can add more later.



 4. More specs - such as naming conversions, common constant reference
 definitions ...

 These are just a draft, not the spec, correct me if I am wrong
 understanding and add the missing aspects, we can discuss these interface
 and data model clearly I think.


 --
 *Haomeng*
 *Thanks:)*



Cheers,
Devananda




 At 2013-11-21 16:08:00,Ladislav Smola lsm...@redhat.com wrote:

 Responses inline.

 On 11/20/2013 07:14 PM, Devananda van der Veen wrote:

 Responses inline.

  On Wed, Nov 20, 2013 at 2:19 AM, Ladislav Smola lsm...@redhat.comwrote:

 Ok, I'll try to summarize what will be done in the near future for
 Undercloud monitoring.

 1. There will be Central agent running on the same host(hosts once the
 central agent horizontal scaling is finished) as Ironic


  Ironic is meant to be run with 1 conductor service. By i-2 milestone we
 should be able to do this, and running at least 2 conductors will be
 recommended. When will Ceilometer be able to run with multiple agents?


 Here it is described and tracked:
 https://blueprints.launchpad.net/ceilometer/+spec/central-agent-improvement


  On a side note, it is a bit confusing to call something a central
 agent if it is meant to be horizontally scaled. The ironic-conductor
 service has been designed to scale out in a similar way to nova-conductor;
 that is, there may be many of them in an AZ. I'm not sure that there is a
 need for Ceilometer's agent to scale in exactly a 1:1 relationship with
 ironic-conductor?


 Yeah we have 

Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-25 Thread Doug Hellmann
On Mon, Nov 25, 2013 at 3:56 PM, Devananda van der Veen 
devananda@gmail.com wrote:

 Hi!

 Very good questions. I think most of them are directed towards the
 Ceilometer team, but I have answered a few bits inline.


 On Mon, Nov 25, 2013 at 7:24 AM, wanghaomeng wanghaom...@163.com wrote:


 Hello all:

 Basically, I understand the solution is - Our Ironic will implement an
 IPMI driver


 We will need to add a new interface -- for example,
 ironic.drivers.base.BaseDriver:sensor and the corresponding
 ironic.drivers.base.SensorInterface class, then implement this interface as
 ironic.drivers.modules.ipmitool:IPMISensor

 We also need to define the methods this interface supports and what the
 return data type is for each method. I imagine it may be something like:
 - SensorInterface.list_available_sensors(node) returns a list of sensor
 names for that node
 - SensorInterface.get_measurements(node, list_of_sensor_names) returns a
 dict of dicts, eg, { 'sensor_1': {'key': 'value'}, 'sensor_2': ...}


 (extendable framework for more drivers) to collect hardware sensor
 data(cpu temp, fan speed, volts, etc) via IPMI protocol from hardware
 server node, and emit the AMQP message to Ceilometer Collector,
 Ceilometer have the framework to handle the valid sample message and save
 to the database for data retrieving by consumer.

 Now, how do you think if we should clearly define the *interface  data
 model *specifications between Ironic and Ceilometer to enable IPMI data
 collecting, then our two team can start the coding together?


 I think this is just a matter of understanding Ceilometer's API so that
 Ironic can emit messages in the correct format. You've got many good
 questions for the Ceilometer team on this below.



 And I still have some concern with our interface and data model as below,
 the spec need to be discussed and finalized:

 1. What is the Ceilometer sample data mandatory attributes, such as
 instance_id/tenant_id/user_id/resource_id, if they are not  optional, where
 are these data populated, from Ironic or Ceilomter side?


   *name/type/unit/volume/timestamp* - basic sample property, can be
 populated from Ironic side as data source
   *user_id/project_id/resource_id* - Ironic or Ceilometer populate these
 fields??


Ceilometer knows nothing about resources unless it is told, so all of the
required fields have to be provided by the sender.



   *resource_metadata - this is for Ceilometer metadata query, Ironic know
 nothing for such resource metadata I think*


The resource metadata depends on the resource type, but should be all of
the user-visible attributes for that object stored in the database at the
time the measurement is taken. For example, for instances we (try to) get
all of the instance attributes.


   *source *- can we hard-code as 'hardware' as a source identifier?


No, the source is the source of the user and project ids, not the source
of the measurement (the data source is implied by the meter name). The
default source for user and project is openstack to differentiate from an
add-on layer like a PaaS where there are different user or project ids.





 Ironic can cache the user_id and project_id of the instance. These will
 not be present for unprovisioned nodes.

 I'm not sure what resource_id is in this context, perhaps the nova
 instance_uuid? If so, Ironic has that as well.


Do end-users know about bare metal servers before they are provisioned as
instances? Can a regular user, for example, as for the list of available
servers or find details about one by name or id?





 2. Not sure if our Ceilometer only accept the *signed-message*, if it is
 case, how Ironic get the message trust for Ceilometer, and send the valid
 message which can be accepted by Ceilometer Collector?


I'm not sure it's appropriate for ironic to be sending messages using
ceilometer's sample format. We receive data from the other projects using
the more generic notification system, and that seems like the right tool to
use here, too. Unless the other ceilometer devs disagree?




 3. What is the Ceilometer sample data structure, and what is the min data
 item set for the IPMI message be emitted to Collector?
   *name/type/unit/volume/**timestamp/source - is this min data item set?*

 3. If the detailed data model should be defined for our IPMI data now?,
 what is our the first version scope, how many IPMI data type we should
 support? Here is a IPMI data sample list, I think we can support these as a
 min set.
   *Temperature - System Temp/CPU Temp*
 *  FAN Speed in rpm - FAN 1/2/3/4/A*
 *  Volts - Vcore/3.3VCC/12V/VDIMM/5VCC/-12V/VBAT/VSB/AVCC*


 I think that's a good starting list. We can add more later.



 4. More specs - such as naming conversions, common constant reference
 definitions ...


 These are just a draft, not the spec, correct me if I am wrong
 understanding and add the missing aspects, we can discuss these interface
 and data model clearly I think.


 

Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-21 Thread Devananda van der Veen
On Thu, Nov 21, 2013 at 12:08 AM, Ladislav Smola lsm...@redhat.com wrote:

  Responses inline.


 On 11/20/2013 07:14 PM, Devananda van der Veen wrote:

 Responses inline.

  On Wed, Nov 20, 2013 at 2:19 AM, Ladislav Smola lsm...@redhat.comwrote:

 Ok, I'll try to summarize what will be done in the near future for
 Undercloud monitoring.

 1. There will be Central agent running on the same host(hosts once the
 central agent horizontal scaling is finished) as Ironic


  Ironic is meant to be run with 1 conductor service. By i-2 milestone we
 should be able to do this, and running at least 2 conductors will be
 recommended. When will Ceilometer be able to run with multiple agents?


 Here it is described and tracked:
 https://blueprints.launchpad.net/ceilometer/+spec/central-agent-improvement


Thanks - I've subscribed to it.


On a side note, it is a bit confusing to call something a central
 agent if it is meant to be horizontally scaled. The ironic-conductor
 service has been designed to scale out in a similar way to nova-conductor;
 that is, there may be many of them in an AZ. I'm not sure that there is a
 need for Ceilometer's agent to scale in exactly a 1:1 relationship with
 ironic-conductor?


 Yeah we have already talked about that. Maybe some renaming will be in
 place later. :-) I don't think it has to be 1:1 mapping. There was only
 requirement to have Hardware agent only on hosts with ironic-conductor,
 so it has access to management network, right?


Correct.

 2. It will have SNMP pollster, SNMP pollster will be able to get list of
 hosts and their IPs from Nova (last time I
 checked it was in Nova) so it can poll them for stats. Hosts to poll
 can be also defined statically in config file.


  Assuming all the undercloud images have an SNMP daemon baked in, which
 they should, then this is fine. And yes, Nova can give you the IP addresses
 for instances provisioned via Ironic.



 Yes.


3. It will have IPMI pollster, that will poll Ironic API, getting list
 of hosts and a fixed set of stats (basically everything
 that we can get :-))


  No -- I thought we just agreed that Ironic will not expose an API for
 IPMI data. You can poll Nova to get a list of instances (that are on bare
 metal) and you can poll Ironic to get a list of nodes (either nodes that
 have an instance associated, or nodes that are unprovisioned) but this will
 only give you basic information about the node (such as the MAC addresses
 of its network ports, and whether it is on/off, etc).


 Ok sorry I have misunderstood the:
 If there is a fixed set of information (eg, temp, fan speed, etc) that
 ceilometer will want,let's make a list of that and add a driver interface
 within Ironic to abstract the collection of that information from physical
 nodes. Then, each driver will be able to implement it as necessary for that
 vendor. Eg., an iLO driver may poll its nodes differently than a generic
 IPMI driver, but the resulting data exported to Ceilometer should have the
 same structure.

 I thought I've read the data will be exposed, but it will be just internal
 Ironic abstraction, that will be polled by Ironic and send directly do
 Ceilometer collector. So same as the point 4., right? Yeah I guess this
 will be easier to implement.


Yes -- you are correct. I was referring to an internal abstraction around
different hardware drivers.





 4. Ironic will also emit messages (basically all events regarding the
 hardware) and send them directly to Ceilometer collector


  Correct. I've updated the BP:

  https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent

  Let me know if that looks like a good description.


 Yeah, seems great. I would maybe remove the word 'Agent', seems Ironic
 will send it directly to Ceilometer collector, so Ironic acts as agent,
 right?


Fair point - I have updated the BP and renamed it to

https://blueprints.launchpad.net/ironic/+spec/send-data-to-ceilometer







 -Devananda



 Does it seems to be correct? I think that is the basic we must have to
 have Undercloud monitored. We can then build on that.

 Kind regards,
 Ladislav



 On 11/20/2013 09:22 AM, Julien Danjou wrote:

 On Tue, Nov 19 2013, Devananda van der Veen wrote:

 If there is a fixed set of information (eg, temp, fan speed, etc) that
 ceilometer will want,

 Sure, we want everything.

 let's make a list of that and add a driver interface
 within Ironic to abstract the collection of that information from
 physical
 nodes. Then, each driver will be able to implement it as necessary for
 that
 vendor. Eg., an iLO driver may poll its nodes differently than a generic
 IPMI driver, but the resulting data exported to Ceilometer should have
 the
 same structure.

 I like the idea.

 An SNMP agent doesn't fit within the scope of Ironic, as far as I see, so
 this would need to be implemented by Ceilometer.

 We're working on adding pollster for that indeed.

 As far as where the SNMP agent would need to run, it 

Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-20 Thread Julien Danjou
On Tue, Nov 19 2013, Devananda van der Veen wrote:

 If there is a fixed set of information (eg, temp, fan speed, etc) that
 ceilometer will want,

Sure, we want everything.

 let's make a list of that and add a driver interface
 within Ironic to abstract the collection of that information from physical
 nodes. Then, each driver will be able to implement it as necessary for that
 vendor. Eg., an iLO driver may poll its nodes differently than a generic
 IPMI driver, but the resulting data exported to Ceilometer should have the
 same structure.

I like the idea.

 An SNMP agent doesn't fit within the scope of Ironic, as far as I see, so
 this would need to be implemented by Ceilometer.

We're working on adding pollster for that indeed.

 As far as where the SNMP agent would need to run, it should be on the
 same host(s) as ironic-conductor so that it has access to the
 management network (the physically-separate network for hardware
 management, IPMI, etc). We should keep the number of applications with
 direct access to that network to a minimum, however, so a thin agent
 that collects and forwards the SNMP data to the central agent would be
 preferable, in my opinion.

We can keep things simple by having the agent only doing that polling I
think. Building a new agent sounds like it will complicate deployment
again.

-- 
Julien Danjou
// Free Software hacker / independent consultant
// http://julien.danjou.info


signature.asc
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-20 Thread Ladislav Smola
Ok, I'll try to summarize what will be done in the near future for 
Undercloud monitoring.


1. There will be Central agent running on the same host(hosts once the 
central agent horizontal scaling is finished) as Ironic
2. It will have SNMP pollster, SNMP pollster will be able to get list of 
hosts and their IPs from Nova (last time I
checked it was in Nova) so it can poll them for stats. Hosts to 
poll can be also defined statically in config file.
3. It will have IPMI pollster, that will poll Ironic API, getting list 
of hosts and a fixed set of stats (basically everything

that we can get :-))
4. Ironic will also emit messages (basically all events regarding the 
hardware) and send them directly to Ceilometer collector


Does it seems to be correct? I think that is the basic we must have to 
have Undercloud monitored. We can then build on that.


Kind regards,
Ladislav

On 11/20/2013 09:22 AM, Julien Danjou wrote:

On Tue, Nov 19 2013, Devananda van der Veen wrote:


If there is a fixed set of information (eg, temp, fan speed, etc) that
ceilometer will want,

Sure, we want everything.


let's make a list of that and add a driver interface
within Ironic to abstract the collection of that information from physical
nodes. Then, each driver will be able to implement it as necessary for that
vendor. Eg., an iLO driver may poll its nodes differently than a generic
IPMI driver, but the resulting data exported to Ceilometer should have the
same structure.

I like the idea.


An SNMP agent doesn't fit within the scope of Ironic, as far as I see, so
this would need to be implemented by Ceilometer.

We're working on adding pollster for that indeed.


As far as where the SNMP agent would need to run, it should be on the
same host(s) as ironic-conductor so that it has access to the
management network (the physically-separate network for hardware
management, IPMI, etc). We should keep the number of applications with
direct access to that network to a minimum, however, so a thin agent
that collects and forwards the SNMP data to the central agent would be
preferable, in my opinion.

We can keep things simple by having the agent only doing that polling I
think. Building a new agent sounds like it will complicate deployment
again.




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-20 Thread Devananda van der Veen
Responses inline.

On Wed, Nov 20, 2013 at 2:19 AM, Ladislav Smola lsm...@redhat.com wrote:

 Ok, I'll try to summarize what will be done in the near future for
 Undercloud monitoring.

 1. There will be Central agent running on the same host(hosts once the
 central agent horizontal scaling is finished) as Ironic


Ironic is meant to be run with 1 conductor service. By i-2 milestone we
should be able to do this, and running at least 2 conductors will be
recommended. When will Ceilometer be able to run with multiple agents?

On a side note, it is a bit confusing to call something a central agent
if it is meant to be horizontally scaled. The ironic-conductor service has
been designed to scale out in a similar way to nova-conductor; that is,
there may be many of them in an AZ. I'm not sure that there is a need for
Ceilometer's agent to scale in exactly a 1:1 relationship with
ironic-conductor?


 2. It will have SNMP pollster, SNMP pollster will be able to get list of
 hosts and their IPs from Nova (last time I
 checked it was in Nova) so it can poll them for stats. Hosts to poll
 can be also defined statically in config file.


Assuming all the undercloud images have an SNMP daemon baked in, which they
should, then this is fine. And yes, Nova can give you the IP addresses for
instances provisioned via Ironic.


 3. It will have IPMI pollster, that will poll Ironic API, getting list of
 hosts and a fixed set of stats (basically everything
 that we can get :-))


No -- I thought we just agreed that Ironic will not expose an API for IPMI
data. You can poll Nova to get a list of instances (that are on bare metal)
and you can poll Ironic to get a list of nodes (either nodes that have an
instance associated, or nodes that are unprovisioned) but this will only
give you basic information about the node (such as the MAC addresses of its
network ports, and whether it is on/off, etc).


 4. Ironic will also emit messages (basically all events regarding the
 hardware) and send them directly to Ceilometer collector


Correct. I've updated the BP:

https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent

Let me know if that looks like a good description.

-Devananda



 Does it seems to be correct? I think that is the basic we must have to
 have Undercloud monitored. We can then build on that.

 Kind regards,
 Ladislav



 On 11/20/2013 09:22 AM, Julien Danjou wrote:

 On Tue, Nov 19 2013, Devananda van der Veen wrote:

  If there is a fixed set of information (eg, temp, fan speed, etc) that
 ceilometer will want,

 Sure, we want everything.

  let's make a list of that and add a driver interface
 within Ironic to abstract the collection of that information from
 physical
 nodes. Then, each driver will be able to implement it as necessary for
 that
 vendor. Eg., an iLO driver may poll its nodes differently than a generic
 IPMI driver, but the resulting data exported to Ceilometer should have
 the
 same structure.

 I like the idea.

  An SNMP agent doesn't fit within the scope of Ironic, as far as I see, so
 this would need to be implemented by Ceilometer.

 We're working on adding pollster for that indeed.

  As far as where the SNMP agent would need to run, it should be on the
 same host(s) as ironic-conductor so that it has access to the
 management network (the physically-separate network for hardware
 management, IPMI, etc). We should keep the number of applications with
 direct access to that network to a minimum, however, so a thin agent
 that collects and forwards the SNMP data to the central agent would be
 preferable, in my opinion.

 We can keep things simple by having the agent only doing that polling I
 think. Building a new agent sounds like it will complicate deployment
 again.



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-19 Thread Doug Hellmann
On Mon, Nov 18, 2013 at 8:58 PM, Lu, Lianhao lianhao...@intel.com wrote:


 Doug Hellmann wrote on 2013-11-19:
 
  On Mon, Nov 18, 2013 at 12:25 PM, Devananda van der Veen 
 devananda@gmail.com wrote:
 
 
Hi Lianhao Lu,
 
I briefly summarized my recollection of that session in this
 blueprint:
 
https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent
 
 
I've responded to your questions inline as well.
 
 
On Sun, Nov 17, 2013 at 10:24 PM, Lu, Lianhao 
 lianhao...@intel.com wrote:
 
 
Hi stackers,
 
During the summit session Expose hardware sensor (IPMI)
 data
 
 https://etherpad.openstack.org/p/icehouse-summit-ceilometer-hardware-sensors,
 it was proposed to deploy a ceilometer agent next to
  the ironic conductor to the get the ipmi data. Here I'd like to ask some
 questions to figure out what's the current missing pieces in ironic
  and ceilometer for that proposal.
 
1. Just double check, ironic won't provide API to get IPMI
 data, right?
 
 
 
Correct. This was generally felt to be unnecessary.
 
 
2. If deploying a ceilometer agent next to the ironic
 conductor, how does the agent talk to the conductor? Through rpc?
 
 
 
My understanding is that ironic-conductor will emit messages to
 the ceilimeter agent, and the communication is one-way. These could
  be triggered by a periodic task, or by some other event within Ironic,
 such as a change in the power state of a node.
 
 
  Cool, so this eliminates the need for a separate agent. The ceilometer
 work can be done in the collector, to receive the new messages.
 
 Does this means we lose the ability to specify the different polling
 interval for different monitoring data, like we have in ceilometer pipeline?


It means that would need to be implemented in the ironic code that is doing
the polling. 

Doug



 
3. Does the current ironic conductor have rpc_method to
 support getting generic ipmi data, i.e. let the rpc_method caller
  specifying arbitrary netfn/command to get any type of ipmi data?
 
 
 
No, and as far as I understand, it doesn't need one.
 
 
  It would only need that if we were going to poll for the data, but if
 ironic is emitting notifications then we don't have to do that.
 
4. I believe the ironic conductor uses some kind of
 node_id to associate the bmc with its credentials, right? If so, how can the
  ceilometer agent get those node_ids to ask the ironic conductor to poll
 the ipmi data? And how can the ceilometer agent extract
  meaningful information from that node_id to set those fields in the
 ceilometer Sample(e.g. recource_id, project_id, user_id, etc.) to identify
  which physical node the ipmi data is coming from?
 
This question perhaps requires a longer answer.
 
Ironic references physical machines (nodes) internally with an
 integer node_id and externally with a standard uuid. When a Nova
  instance is created, it will be associated to a node, that node will
 have a reference to the nova instance_uuid which is exposed in our API,
  and can be passed to Ceilometer's agent. I believe that nova
 instance_uuid will enable ceilometer to detect the project, user, etc.
 
 
  If ironic has those values (at least the project), and can include them
 in the notification payload, that will make processing the incoming
  notifications significantly less expensive.
 
 
Should Ironic emit messages regarding nodes which are not
 provisioned? Physical machines that don't have a tenant instance on them
  are not associated to any project, user, tenant, quota, etc, so I
 suspect that we shouldn't notify about them. It would be like tracking the
  unused disks in a SAN.
 
 
  I don't think it would be useful, but if someone else does then it seems
 OK to include them.

 I think it'd better for Ironic to emit those data in case some users want
 to collect them, at least Ironic should have a configuration setting to
 emit those kind of data.

 -Lianhao



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-19 Thread Devananda van der Veen
On Mon, Nov 18, 2013 at 10:35 AM, Ladislav Smola lsm...@redhat.com wrote:

  Hello. I have a couple of additional questions.

 1. What about IPMI data that we want to get by polling. E.g. temperatures,
 etc. Will the Ironic be polling these kind of
 data and send them directly to collector(or agent)? Not sure if this
 belongs to Ironic. It would have to support some
 pluggable architecture for vendor specific pollsters like Ceilometer.


If there is a fixed set of information (eg, temp, fan speed, etc) that
ceilometer will want, let's make a list of that and add a driver interface
within Ironic to abstract the collection of that information from physical
nodes. Then, each driver will be able to implement it as necessary for that
vendor. Eg., an iLO driver may poll its nodes differently than a generic
IPMI driver, but the resulting data exported to Ceilometer should have the
same structure.

I don't think we should, at least right now, support pluggable pollsters on
the Ceilometer-Ironic side. Let's start with a small set of data that
Ironic exposes, make it pluggable internally for different types of
hardware, and iterate if necessary.


 2. I've seen in the etherpad that the SNMP agent(pollster) will be also
 part of the Ironic(running next to conductor). Is it true?
 Or that will be placed in Ceilometer central agent?


An SNMP agent doesn't fit within the scope of Ironic, as far as I see, so
this would need to be implemented by Ceilometer.

As far as where the SNMP agent would need to run, it should be on the same
host(s) as ironic-conductor so that it has access to the management network
(the physically-separate network for hardware management, IPMI, etc). We
should keep the number of applications with direct access to that network
to a minimum, however, so a thin agent that collects and forwards the SNMP
data to the central agent would be preferable, in my opinion.


Regards,
Devananda





 Thanks for response.
 Ladislav



 On 11/18/2013 06:25 PM, Devananda van der Veen wrote:

 Hi Lianhao Lu,

  I briefly summarized my recollection of that session in this blueprint:

  https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent

  I've responded to your questions inline as well.


 On Sun, Nov 17, 2013 at 10:24 PM, Lu, Lianhao lianhao...@intel.comwrote:

 Hi stackers,

 During the summit session Expose hardware sensor (IPMI) data
 https://etherpad.openstack.org/p/icehouse-summit-ceilometer-hardware-sensors,
 it was proposed to deploy a ceilometer agent next to the ironic conductor
 to the get the ipmi data. Here I'd like to ask some questions to figure out
 what's the current missing pieces in ironic and ceilometer for that
 proposal.

 1. Just double check, ironic won't provide API to get IPMI data, right?


  Correct. This was generally felt to be unnecessary.


 2. If deploying a ceilometer agent next to the ironic conductor, how does
 the agent talk to the conductor? Through rpc?


  My understanding is that ironic-conductor will emit messages to the
 ceilimeter agent, and the communication is one-way. These could be
 triggered by a periodic task, or by some other event within Ironic, such as
 a change in the power state of a node.



 3. Does the current ironic conductor have rpc_method to support getting
 generic ipmi data, i.e. let the rpc_method caller specifying arbitrary
 netfn/command to get any type of ipmi data?


  No, and as far as I understand, it doesn't need one.



 4. I believe the ironic conductor uses some kind of node_id to associate
 the bmc with its credentials, right? If so, how can the ceilometer agent
 get those node_ids to ask the ironic conductor to poll the ipmi data? And
 how can the ceilometer agent extract meaningful information from that
 node_id to set those fields in the ceilometer Sample(e.g. recource_id,
 project_id, user_id, etc.) to identify which physical node the ipmi data is
 coming from?


  This question perhaps requires a longer answer.

  Ironic references physical machines (nodes) internally with an integer
 node_id and externally with a standard uuid. When a Nova instance is
 created, it will be associated to a node, that node will have a reference
 to the nova instance_uuid which is exposed in our API, and can be passed to
 Ceilometer's agent. I believe that nova instance_uuid will enable
 ceilometer to detect the project, user, etc.

  Should Ironic emit messages regarding nodes which are not provisioned?
 Physical machines that don't have a tenant instance on them are not
 associated to any project, user, tenant, quota, etc, so I suspect that we
 shouldn't notify about them. It would be like tracking the unused disks in
 a SAN.

  Regards,
 Devananda



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-18 Thread Devananda van der Veen
Hi Lianhao Lu,

I briefly summarized my recollection of that session in this blueprint:

https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent

I've responded to your questions inline as well.


On Sun, Nov 17, 2013 at 10:24 PM, Lu, Lianhao lianhao...@intel.com wrote:

 Hi stackers,

 During the summit session Expose hardware sensor (IPMI) data
 https://etherpad.openstack.org/p/icehouse-summit-ceilometer-hardware-sensors,
 it was proposed to deploy a ceilometer agent next to the ironic conductor
 to the get the ipmi data. Here I'd like to ask some questions to figure out
 what's the current missing pieces in ironic and ceilometer for that
 proposal.

 1. Just double check, ironic won't provide API to get IPMI data, right?


Correct. This was generally felt to be unnecessary.


 2. If deploying a ceilometer agent next to the ironic conductor, how does
 the agent talk to the conductor? Through rpc?


My understanding is that ironic-conductor will emit messages to the
ceilimeter agent, and the communication is one-way. These could be
triggered by a periodic task, or by some other event within Ironic, such as
a change in the power state of a node.



 3. Does the current ironic conductor have rpc_method to support getting
 generic ipmi data, i.e. let the rpc_method caller specifying arbitrary
 netfn/command to get any type of ipmi data?


No, and as far as I understand, it doesn't need one.



 4. I believe the ironic conductor uses some kind of node_id to associate
 the bmc with its credentials, right? If so, how can the ceilometer agent
 get those node_ids to ask the ironic conductor to poll the ipmi data? And
 how can the ceilometer agent extract meaningful information from that
 node_id to set those fields in the ceilometer Sample(e.g. recource_id,
 project_id, user_id, etc.) to identify which physical node the ipmi data is
 coming from?


This question perhaps requires a longer answer.

Ironic references physical machines (nodes) internally with an integer
node_id and externally with a standard uuid. When a Nova instance is
created, it will be associated to a node, that node will have a reference
to the nova instance_uuid which is exposed in our API, and can be passed to
Ceilometer's agent. I believe that nova instance_uuid will enable
ceilometer to detect the project, user, etc.

Should Ironic emit messages regarding nodes which are not provisioned?
Physical machines that don't have a tenant instance on them are not
associated to any project, user, tenant, quota, etc, so I suspect that we
shouldn't notify about them. It would be like tracking the unused disks in
a SAN.

Regards,
Devananda
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-18 Thread Doug Hellmann
On Mon, Nov 18, 2013 at 12:25 PM, Devananda van der Veen 
devananda@gmail.com wrote:

 Hi Lianhao Lu,

 I briefly summarized my recollection of that session in this blueprint:

 https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent

 I've responded to your questions inline as well.


 On Sun, Nov 17, 2013 at 10:24 PM, Lu, Lianhao lianhao...@intel.comwrote:

 Hi stackers,

 During the summit session Expose hardware sensor (IPMI) data
 https://etherpad.openstack.org/p/icehouse-summit-ceilometer-hardware-sensors,
 it was proposed to deploy a ceilometer agent next to the ironic conductor
 to the get the ipmi data. Here I'd like to ask some questions to figure out
 what's the current missing pieces in ironic and ceilometer for that
 proposal.

 1. Just double check, ironic won't provide API to get IPMI data, right?


 Correct. This was generally felt to be unnecessary.


 2. If deploying a ceilometer agent next to the ironic conductor, how does
 the agent talk to the conductor? Through rpc?


 My understanding is that ironic-conductor will emit messages to the
 ceilimeter agent, and the communication is one-way. These could be
 triggered by a periodic task, or by some other event within Ironic, such as
 a change in the power state of a node.


Cool, so this eliminates the need for a separate agent. The ceilometer
work can be done in the collector, to receive the new messages.






 3. Does the current ironic conductor have rpc_method to support getting
 generic ipmi data, i.e. let the rpc_method caller specifying arbitrary
 netfn/command to get any type of ipmi data?


 No, and as far as I understand, it doesn't need one.


It would only need that if we were going to poll for the data, but if
ironic is emitting notifications then we don't have to do that.






 4. I believe the ironic conductor uses some kind of node_id to associate
 the bmc with its credentials, right? If so, how can the ceilometer agent
 get those node_ids to ask the ironic conductor to poll the ipmi data? And
 how can the ceilometer agent extract meaningful information from that
 node_id to set those fields in the ceilometer Sample(e.g. recource_id,
 project_id, user_id, etc.) to identify which physical node the ipmi data is
 coming from?


 This question perhaps requires a longer answer.

 Ironic references physical machines (nodes) internally with an integer
 node_id and externally with a standard uuid. When a Nova instance is
 created, it will be associated to a node, that node will have a reference
 to the nova instance_uuid which is exposed in our API, and can be passed to
 Ceilometer's agent. I believe that nova instance_uuid will enable
 ceilometer to detect the project, user, etc.


If ironic has those values (at least the project), and can include them in
the notification payload, that will make processing the incoming
notifications significantly less expensive. 



 Should Ironic emit messages regarding nodes which are not provisioned?
 Physical machines that don't have a tenant instance on them are not
 associated to any project, user, tenant, quota, etc, so I suspect that we
 shouldn't notify about them. It would be like tracking the unused disks in
 a SAN.


I don't think it would be useful, but if someone else does then it seems OK
to include them.

Doug




 Regards,
 Devananda


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-18 Thread Devananda van der Veen
On Mon, Nov 18, 2013 at 9:40 AM, Doug Hellmann
doug.hellm...@dreamhost.comwrote:


 On Mon, Nov 18, 2013 at 12:25 PM, Devananda van der Veen 
 devananda@gmail.com wrote:

 Hi Lianhao Lu,

 I briefly summarized my recollection of that session in this blueprint:

 https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent

 I've responded to your questions inline as well.


 On Sun, Nov 17, 2013 at 10:24 PM, Lu, Lianhao lianhao...@intel.comwrote:

 Hi stackers,

 During the summit session Expose hardware sensor (IPMI) data
 https://etherpad.openstack.org/p/icehouse-summit-ceilometer-hardware-sensors,
 it was proposed to deploy a ceilometer agent next to the ironic conductor
 to the get the ipmi data. Here I'd like to ask some questions to figure out
 what's the current missing pieces in ironic and ceilometer for that
 proposal.

 1. Just double check, ironic won't provide API to get IPMI data, right?


 Correct. This was generally felt to be unnecessary.


 2. If deploying a ceilometer agent next to the ironic conductor, how
 does the agent talk to the conductor? Through rpc?


 My understanding is that ironic-conductor will emit messages to the
 ceilimeter agent, and the communication is one-way. These could be
 triggered by a periodic task, or by some other event within Ironic, such as
 a change in the power state of a node.


 Cool, so this eliminates the need for a separate agent. The ceilometer
 work can be done in the collector, to receive the new messages.






 3. Does the current ironic conductor have rpc_method to support getting
 generic ipmi data, i.e. let the rpc_method caller specifying arbitrary
 netfn/command to get any type of ipmi data?


  No, and as far as I understand, it doesn't need one.


 It would only need that if we were going to poll for the data, but if
 ironic is emitting notifications then we don't have to do that.






 4. I believe the ironic conductor uses some kind of node_id to associate
 the bmc with its credentials, right? If so, how can the ceilometer agent
 get those node_ids to ask the ironic conductor to poll the ipmi data? And
 how can the ceilometer agent extract meaningful information from that
 node_id to set those fields in the ceilometer Sample(e.g. recource_id,
 project_id, user_id, etc.) to identify which physical node the ipmi data is
 coming from?


 This question perhaps requires a longer answer.

 Ironic references physical machines (nodes) internally with an integer
 node_id and externally with a standard uuid. When a Nova instance is
 created, it will be associated to a node, that node will have a reference
 to the nova instance_uuid which is exposed in our API, and can be passed to
 Ceilometer's agent. I believe that nova instance_uuid will enable
 ceilometer to detect the project, user, etc.


 If ironic has those values (at least the project), and can include them in
 the notification payload, that will make processing the incoming
 notifications significantly less expensive.


We can take a look as we get closer to completing this. Ironic doesn't know
anything about projects today, but we may be able to pass the project_id in
from Nova and cache it as a node property for the duration of the instance,
if that's an important optimization.

-D
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-18 Thread Doug Hellmann
On Mon, Nov 18, 2013 at 1:35 PM, Ladislav Smola lsm...@redhat.com wrote:

  Hello. I have a couple of additional questions.

 1. What about IPMI data that we want to get by polling. E.g. temperatures,
 etc. Will the Ironic be polling these kind of
 data and send them directly to collector(or agent)? Not sure if this
 belongs to Ironic. It would have to support some
 pluggable architecture for vendor specific pollsters like Ceilometer.


The ceilometer and ironic teams will need to collaborate to ensure the
desired data is being collected.



 2. I've seen in the etherpad that the SNMP agent(pollster) will be also
 part of the Ironic(running next to conductor). Is it true?
 Or that will be placed in Ceilometer central agent?


Good question.

Doug





 Thanks for response.
 Ladislav



 On 11/18/2013 06:25 PM, Devananda van der Veen wrote:

 Hi Lianhao Lu,

  I briefly summarized my recollection of that session in this blueprint:

  https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent

  I've responded to your questions inline as well.


 On Sun, Nov 17, 2013 at 10:24 PM, Lu, Lianhao lianhao...@intel.comwrote:

 Hi stackers,

 During the summit session Expose hardware sensor (IPMI) data
 https://etherpad.openstack.org/p/icehouse-summit-ceilometer-hardware-sensors,
 it was proposed to deploy a ceilometer agent next to the ironic conductor
 to the get the ipmi data. Here I'd like to ask some questions to figure out
 what's the current missing pieces in ironic and ceilometer for that
 proposal.

 1. Just double check, ironic won't provide API to get IPMI data, right?


  Correct. This was generally felt to be unnecessary.


 2. If deploying a ceilometer agent next to the ironic conductor, how does
 the agent talk to the conductor? Through rpc?


  My understanding is that ironic-conductor will emit messages to the
 ceilimeter agent, and the communication is one-way. These could be
 triggered by a periodic task, or by some other event within Ironic, such as
 a change in the power state of a node.



 3. Does the current ironic conductor have rpc_method to support getting
 generic ipmi data, i.e. let the rpc_method caller specifying arbitrary
 netfn/command to get any type of ipmi data?


  No, and as far as I understand, it doesn't need one.



 4. I believe the ironic conductor uses some kind of node_id to associate
 the bmc with its credentials, right? If so, how can the ceilometer agent
 get those node_ids to ask the ironic conductor to poll the ipmi data? And
 how can the ceilometer agent extract meaningful information from that
 node_id to set those fields in the ceilometer Sample(e.g. recource_id,
 project_id, user_id, etc.) to identify which physical node the ipmi data is
 coming from?


  This question perhaps requires a longer answer.

  Ironic references physical machines (nodes) internally with an integer
 node_id and externally with a standard uuid. When a Nova instance is
 created, it will be associated to a node, that node will have a reference
 to the nova instance_uuid which is exposed in our API, and can be passed to
 Ceilometer's agent. I believe that nova instance_uuid will enable
 ceilometer to detect the project, user, etc.

  Should Ironic emit messages regarding nodes which are not provisioned?
 Physical machines that don't have a tenant instance on them are not
 associated to any project, user, tenant, quota, etc, so I suspect that we
 shouldn't notify about them. It would be like tracking the unused disks in
 a SAN.

  Regards,
 Devananda



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer

2013-11-18 Thread Lu, Lianhao

Doug Hellmann wrote on 2013-11-19:
 
 On Mon, Nov 18, 2013 at 12:25 PM, Devananda van der Veen 
 devananda@gmail.com wrote:
 
 
   Hi Lianhao Lu,
 
   I briefly summarized my recollection of that session in this blueprint:
 
   https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent
 
 
   I've responded to your questions inline as well.
 
 
   On Sun, Nov 17, 2013 at 10:24 PM, Lu, Lianhao lianhao...@intel.com 
 wrote:
 
 
   Hi stackers,
 
   During the summit session Expose hardware sensor (IPMI) data
 https://etherpad.openstack.org/p/icehouse-summit-ceilometer-hardware-sensors, 
 it was proposed to deploy a ceilometer agent next to
 the ironic conductor to the get the ipmi data. Here I'd like to ask some 
 questions to figure out what's the current missing pieces in ironic
 and ceilometer for that proposal.
 
   1. Just double check, ironic won't provide API to get IPMI 
 data, right?
 
 
 
   Correct. This was generally felt to be unnecessary.
 
 
   2. If deploying a ceilometer agent next to the ironic 
 conductor, how does the agent talk to the conductor? Through rpc?
 
 
 
   My understanding is that ironic-conductor will emit messages to the 
 ceilimeter agent, and the communication is one-way. These could
 be triggered by a periodic task, or by some other event within Ironic, such 
 as a change in the power state of a node.
 
 
 Cool, so this eliminates the need for a separate agent. The ceilometer work 
 can be done in the collector, to receive the new messages.
 
Does this means we lose the ability to specify the different polling interval 
for different monitoring data, like we have in ceilometer pipeline? 

 
   3. Does the current ironic conductor have rpc_method to support 
 getting generic ipmi data, i.e. let the rpc_method caller
 specifying arbitrary netfn/command to get any type of ipmi data?
 
 
 
   No, and as far as I understand, it doesn't need one.
 
 
 It would only need that if we were going to poll for the data, but if ironic 
 is emitting notifications then we don't have to do that.
 
   4. I believe the ironic conductor uses some kind of node_id to 
 associate the bmc with its credentials, right? If so, how can the
 ceilometer agent get those node_ids to ask the ironic conductor to poll the 
 ipmi data? And how can the ceilometer agent extract
 meaningful information from that node_id to set those fields in the 
 ceilometer Sample(e.g. recource_id, project_id, user_id, etc.) to identify
 which physical node the ipmi data is coming from?
 
   This question perhaps requires a longer answer.
 
   Ironic references physical machines (nodes) internally with an integer 
 node_id and externally with a standard uuid. When a Nova
 instance is created, it will be associated to a node, that node will have a 
 reference to the nova instance_uuid which is exposed in our API,
 and can be passed to Ceilometer's agent. I believe that nova instance_uuid 
 will enable ceilometer to detect the project, user, etc.
 
 
 If ironic has those values (at least the project), and can include them in 
 the notification payload, that will make processing the incoming
 notifications significantly less expensive.
 
 
   Should Ironic emit messages regarding nodes which are not provisioned? 
 Physical machines that don't have a tenant instance on them
 are not associated to any project, user, tenant, quota, etc, so I suspect 
 that we shouldn't notify about them. It would be like tracking the
 unused disks in a SAN.
 
 
 I don't think it would be useful, but if someone else does then it seems OK 
 to include them.

I think it'd better for Ironic to emit those data in case some users want to 
collect them, at least Ironic should have a configuration setting to emit those 
kind of data.

-Lianhao



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev