Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
Hi guys, The blueprint is registered : https://blueprints.launchpad.net/ceilometer/+spec/hbase-storage-backend, Any idea what's the next step ? :) Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
On Thu, Nov 15 2012, shengjie_...@dell.com wrote: The blueprint is registered : https://blueprints.launchpad.net/ceilometer/+spec/hbase-storage-backend, Any idea what's the next step ? :) Implementing? :) And eventually adding more blueprint to depends on if you can't implement this one directly because of some missing stuff. -- Julien Danjou /* Free Software hacker freelance http://julien.danjou.info */ pgp8jDsQh0QEc.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
Do you plan to implement it? Per my understanding, If yes, then begin implementation, and don't forget to add something like for bp in your patch. Otherwise, wait someone who has interests to work on it :) Thanks --jyh -Original Message- From: openstack-bounces+yunhong.jiang=intel@lists.launchpad.net [mailto:openstack-bounces+yunhong.jiang=intel@lists.launchpad.net] On Behalf Of shengjie_...@dell.com Sent: Thursday, November 15, 2012 10:11 PM To: doug.hellm...@dreamhost.com Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint? Hi guys, The blueprint is registered : https://blueprints.launchpad.net/ceilometer/+spec/hbase-storage-backend, Any idea what's the next step ? J Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
Implementing? :) And eventually adding more blueprint to depends on if you can't implement this one directly because of some missing stuff. Two questions: 1. Does it need to be approved first, what's the process to get it approved/assigned to a release? Btw, I was trying to add Julien in the approver list as well, didn't manage to do that. 2. The community meeting normally is the place to have this kind of tech discussion or it's for something else? Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
On Thu, Nov 15 2012, shengjie_...@dell.com wrote: 1. Does it need to be approved first, what's the process to get it approved/assigned to a release? Btw, I was trying to add Julien in the approver list as well, didn't manage to do that. I don't think we've been anything formal so far about that. The storage part has been abstracted so everybody can come up with a new storage engine. So you're planning to work a new one, I don't see any reason not to approve such a blueprint. But I've set this blueprint to approved anyway. :) More discussion is likely to be needed if you need to do modifications to the abstraction layer itself. 2. The community meeting normally is the place to have this kind of tech discussion or it's for something else? It its, so I suggest you bring this today (in 2 minutes) in the open discussion at the end of the meeting, or add it the agenda for next meeting next week. -- Julien Danjou ;; Free Software hacker freelance ;; http://julien.danjou.info pgpcjsYblpraH.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
Thanks Julien, I will write up a blueprint soon. Shengjie -Original Message- From: Julien Danjou [mailto:jul...@danjou.info] Sent: 07 November 2012 18:12 To: Min, Shengjie Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint? On Wed, Nov 07 2012, shengjie_...@dell.com wrote: Hi Shengjie, I am looking for a way to propose a blueprint to the ceilometer team - Adding Hbase storage backend for ceilometer. Currently we have only MongoDB and SQLalchemy as the possible ceilometer storage backend. Given the storage interface is clearly designed and allows us to have different backend, it would be nice to have more options which make Ceilometer more adoptive to some providers' existing architectures. As a cloud IAAS provider, Ceilometer might not be just considered as a usage db, also it's being seen as a centralized place to store all the usage data cross platforms. Additionally, all the historical usage data stored can be used for historical usage analysis as well, this is where MapReduce comes into play. With Hadoop Hbase, it allows usage data to be stored in HDFS, also it gives us the ability to run large scale massive MapRed analysis jobs in the future. This is indeed a good idea, and as you pointed out, Ceilometer collector has been designed so this is being possible. I think you can create a blueprint here: https://blueprints.launchpad.net/ceilometer And write in what should be done. -- Julien Danjou ;; Free Software hacker freelance ;; http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
+1 It will be good to see if the storage API design works for other backends like HBase. Doug On Thu, Nov 8, 2012 at 5:50 AM, shengjie_...@dell.com wrote: Thanks Julien, I will write up a blueprint soon. Shengjie -Original Message- From: Julien Danjou [mailto:jul...@danjou.info] Sent: 07 November 2012 18:12 To: Min, Shengjie Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint? On Wed, Nov 07 2012, shengjie_...@dell.com wrote: Hi Shengjie, I am looking for a way to propose a blueprint to the ceilometer team - Adding Hbase storage backend for ceilometer. Currently we have only MongoDB and SQLalchemy as the possible ceilometer storage backend. Given the storage interface is clearly designed and allows us to have different backend, it would be nice to have more options which make Ceilometer more adoptive to some providers' existing architectures. As a cloud IAAS provider, Ceilometer might not be just considered as a usage db, also it's being seen as a centralized place to store all the usage data cross platforms. Additionally, all the historical usage data stored can be used for historical usage analysis as well, this is where MapReduce comes into play. With Hadoop Hbase, it allows usage data to be stored in HDFS, also it gives us the ability to run large scale massive MapRed analysis jobs in the future. This is indeed a good idea, and as you pointed out, Ceilometer collector has been designed so this is being possible. I think you can create a blueprint here: https://blueprints.launchpad.net/ceilometer And write in what should be done. -- Julien Danjou ;; Free Software hacker freelance ;; http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
Hi Doug, +1 It will be good to see if the storage API design works for other backends like HBase. The first glance I had on ceilometer/storage/base.py, the design of basic storage class does look ok to work with. The only challenge is that the Hbase schema is not as flexible as traditional RDBMS or MongoDB, so the efforts would be more on the Hbase implementation side. Eg. the design of row key, column family etc. We can discuss more once I had a blueprint draft registered :) -- Thanks, Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
On Thu, Nov 8, 2012 at 9:33 AM, shengjie_...@dell.com wrote: Hi Doug, ** ** +1 It will be good to see if the storage API design works for other backends like HBase. ** ** The first glance I had on ceilometer/storage/base.py, the design of basic storage class does look ok to work with. ** ** The only challenge is that the Hbase schema is not as flexible as traditional RDBMS or MongoDB, so the efforts would be more on the Hbase implementation side. Eg. the design of row key, column family etc. I hope that the API provides enough encapsulation to let you implement that in a reasonable way. ** ** We can discuss more once I had a blueprint draft registered J I'm looking forward to it! Doug ** ** -- Thanks, Shengjie ** ** ** ** ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
Hi, I am looking for a way to propose a blueprint to the ceilometer team - Adding Hbase storage backend for ceilometer. Currently we have only MongoDB and SQLalchemy as the possible ceilometer storage backend. Given the storage interface is clearly designed and allows us to have different backend, it would be nice to have more options which make Ceilometer more adoptive to some providers' existing architectures. As a cloud IAAS provider, Ceilometer might not be just considered as a usage db, also it's being seen as a centralized place to store all the usage data cross platforms. Additionally, all the historical usage data stored can be used for historical usage analysis as well, this is where MapReduce comes into play. With Hadoop Hbase, it allows usage data to be stored in HDFS, also it gives us the ability to run large scale massive MapRed analysis jobs in the future. Thanks, Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
On Wed, Nov 07 2012, shengjie_...@dell.com wrote: Hi Shengjie, I am looking for a way to propose a blueprint to the ceilometer team - Adding Hbase storage backend for ceilometer. Currently we have only MongoDB and SQLalchemy as the possible ceilometer storage backend. Given the storage interface is clearly designed and allows us to have different backend, it would be nice to have more options which make Ceilometer more adoptive to some providers' existing architectures. As a cloud IAAS provider, Ceilometer might not be just considered as a usage db, also it's being seen as a centralized place to store all the usage data cross platforms. Additionally, all the historical usage data stored can be used for historical usage analysis as well, this is where MapReduce comes into play. With Hadoop Hbase, it allows usage data to be stored in HDFS, also it gives us the ability to run large scale massive MapRed analysis jobs in the future. This is indeed a good idea, and as you pointed out, Ceilometer collector has been designed so this is being possible. I think you can create a blueprint here: https://blueprints.launchpad.net/ceilometer And write in what should be done. -- Julien Danjou ;; Free Software hacker freelance ;; http://julien.danjou.info pgp0Y6makV0xI.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
