Re: [Openstack] [Metering] Adding a source notion to the schema
On Fri, May 4, 2012 at 12:29 PM, Nick Barcet nick.bar...@canonical.comwrote: Following up on the discussion on IRC yesterday during the metering meeting, I'd like to explain my proposal to add the notion of source to the schema of our event. The current goal we have for ceilometer is to provide a common way to accumulate counters/meters from various openstack component in a central repository that can be then used by other tools to produce bills in fine. One thing we can currently assume in Essex is that all components share a common repository for identity management: Keystone. This is the assumption we made when defining the existing schema. However, I think we need to plan a little further ahead here, and allow for this not to necessarily remain the same in some complex deployment cases. For example, it could be envisioned that some software, outside of the OpenStack project, maybe deployed and used on top of an OpenStack deployment. This software may need to be billed to customers, and collecting meters for it maybe as important for the provider than doing it for OpenStack components. It cannot be assumed that the identity management used by the software will be keystone. The software may not provide a metering interface built in, and it would make sense for the provider to be able to implement this using existing tool present in the underlying OpenStack deployment: ceilometer. In order to allow for this I think it would make sense to: * extend the current schema to allow for an additional source field in the event record. This should be a short identifier. That makes sense. It seems like the identifier(s) used are meant to be defined by the deployer. Is that your intent? * add another record definition that maps source to identity managment URL location * Collecting agent would be in charge to specify the source they map to (keystone by default). It is not clear why the identity management location needs to be recorded in ceilometer. If the deployer controls the source strings, they know what each value means. The only thing that needs to translate the (source, project, user) values to a billing identity is the end-consumer of all of this data, which is also under the control of the deployer. Couldn't the mapping of the source token to the identity location be handled in that layer? I think this would allow for easy extension of ceilometer to support other software, but I'd like to know if you share the usefulness of this extension from your experiences. Thanks, Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Adding a source notion to the schema
On 05/04/2012 02:25 PM, Doug Hellmann wrote: On Fri, May 4, 2012 at 12:29 PM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: Following up on the discussion on IRC yesterday during the metering meeting, I'd like to explain my proposal to add the notion of source to the schema of our event. The current goal we have for ceilometer is to provide a common way to accumulate counters/meters from various openstack component in a central repository that can be then used by other tools to produce bills in fine. One thing we can currently assume in Essex is that all components share a common repository for identity management: Keystone. This is the assumption we made when defining the existing schema. However, I think we need to plan a little further ahead here, and allow for this not to necessarily remain the same in some complex deployment cases. For example, it could be envisioned that some software, outside of the OpenStack project, maybe deployed and used on top of an OpenStack deployment. This software may need to be billed to customers, and collecting meters for it maybe as important for the provider than doing it for OpenStack components. It cannot be assumed that the identity management used by the software will be keystone. The software may not provide a metering interface built in, and it would make sense for the provider to be able to implement this using existing tool present in the underlying OpenStack deployment: ceilometer. In order to allow for this I think it would make sense to: * extend the current schema to allow for an additional source field in the event record. This should be a short identifier. That makes sense. It seems like the identifier(s) used are meant to be defined by the deployer. Is that your intent? Correct. We'll just need a sane default for Keystone. * add another record definition that maps source to identity managment URL location * Collecting agent would be in charge to specify the source they map to (keystone by default). It is not clear why the identity management location needs to be recorded in ceilometer. If the deployer controls the source strings, they know what each value means. The only thing that needs to translate the (source, project, user) values to a billing identity is the end-consumer of all of this data, which is also under the control of the deployer. Couldn't the mapping of the source token to the identity location be handled in that layer? True. It might be over-engineering to try to keep the info in the same place as well, but the impact on the system would be negligible. I'd be happy either ways :) Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Adding a source notion to the schema
On Fri, May 4, 2012 at 6:03 PM, Nick Barcet nick.bar...@canonical.comwrote: On 05/04/2012 02:25 PM, Doug Hellmann wrote: On Fri, May 4, 2012 at 12:29 PM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: Following up on the discussion on IRC yesterday during the metering meeting, I'd like to explain my proposal to add the notion of source to the schema of our event. The current goal we have for ceilometer is to provide a common way to accumulate counters/meters from various openstack component in a central repository that can be then used by other tools to produce bills in fine. One thing we can currently assume in Essex is that all components share a common repository for identity management: Keystone. This is the assumption we made when defining the existing schema. However, I think we need to plan a little further ahead here, and allow for this not to necessarily remain the same in some complex deployment cases. For example, it could be envisioned that some software, outside of the OpenStack project, maybe deployed and used on top of an OpenStack deployment. This software may need to be billed to customers, and collecting meters for it maybe as important for the provider than doing it for OpenStack components. It cannot be assumed that the identity management used by the software will be keystone. The software may not provide a metering interface built in, and it would make sense for the provider to be able to implement this using existing tool present in the underlying OpenStack deployment: ceilometer. In order to allow for this I think it would make sense to: * extend the current schema to allow for an additional source field in the event record. This should be a short identifier. That makes sense. It seems like the identifier(s) used are meant to be defined by the deployer. Is that your intent? Correct. We'll just need a sane default for Keystone. You're focusing on source as the origin for the identity information, but it seems like a layer sitting on top of OpenStack may well use Keystone for identity but generate different billable events. Should source be the origin of the metric data being sent to ceilometer? * add another record definition that maps source to identity managment URL location * Collecting agent would be in charge to specify the source they map to (keystone by default). It is not clear why the identity management location needs to be recorded in ceilometer. If the deployer controls the source strings, they know what each value means. The only thing that needs to translate the (source, project, user) values to a billing identity is the end-consumer of all of this data, which is also under the control of the deployer. Couldn't the mapping of the source token to the identity location be handled in that layer? True. It might be over-engineering to try to keep the info in the same place as well, but the impact on the system would be negligible. I'd be happy either ways :) It will be easier to add it later if we really need it than to take it out if we decide we don't. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Adding a source notion to the schema
On 05/04/2012 03:11 PM, Doug Hellmann wrote: On Fri, May 4, 2012 at 6:03 PM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: On 05/04/2012 02:25 PM, Doug Hellmann wrote: On Fri, May 4, 2012 at 12:29 PM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com [...] In order to allow for this I think it would make sense to: * extend the current schema to allow for an additional source field in the event record. This should be a short identifier. That makes sense. It seems like the identifier(s) used are meant to be defined by the deployer. Is that your intent? Correct. We'll just need a sane default for Keystone. You're focusing on source as the origin for the identity information, but it seems like a layer sitting on top of OpenStack may well use Keystone for identity but generate different billable events. Should source be the origin of the metric data being sent to ceilometer? As I see it, source should remain the same as long as you can correlate project_id and user_id from the same source. * add another record definition that maps source to identity managment URL location * Collecting agent would be in charge to specify the source they map to (keystone by default). It is not clear why the identity management location needs to be recorded in ceilometer. If the deployer controls the source strings, they know what each value means. The only thing that needs to translate the (source, project, user) values to a billing identity is the end-consumer of all of this data, which is also under the control of the deployer. Couldn't the mapping of the source token to the identity location be handled in that layer? True. It might be over-engineering to try to keep the info in the same place as well, but the impact on the system would be negligible. I'd be happy either ways :) It will be easier to add it later if we really need it than to take it out if we decide we don't. Agreed. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
