[Openstack] [ceilometer] HBase Configuration/Instruction is now available on openstack docs
Hi guys, For those of you who wanted to try out HBase as the backend for Ceilometer. Glad to let you know the document for HBase configuration is available now on openstack docs: http://docs.openstack.org/developer/ceilometer/configuration.html#hbase If you had any issues or questions, feel free to ping me on IRC or drop me emails :) All the best, Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Ceilometer] Ganglia Ceilometer integration
If Ganglia has interest in ceilometer data, we can achieve it through a specific publisher, to put data into Gangalia. @yunhong, So it is more like the approach to CW or Synaps, isn't it? if the monitoring system is interested in the meters from Ceilometer, we can have different publishers to push data to different destinations(I think this can be a potential blueprint for ganglia). But that being said, if the backend is something like Cassandra, Mongo or even Hbase, the system metrics(from gmond) and openstack specific meters might be stored in a centralized storage just with different namespaces. Thoughts? - Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Ceilometer] Ganglia Ceilometer integration
Hi guys, Just out of curiosity, Have we ever had any previous threads discussing about possibility of integrating Ceilometer with ganglia as a monitoring approach? Didn't manage to find any from the mailing list archive. Thanks, Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Ceilometer] Ganglia Ceilometer integration
Hi Julien, Just look at the blueprints and your post, seems like there is some effort around synaps, also something around heat, cloudwatch. Just trying to get a clear picture about the vision and direction around monitoring area. Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] How does ceilometer work with multi-DC scenario
Two regions case: If you want to get the meters for TenantA from 'swift-cluster-1' specifically. According to the API design we have now, there is no API can do that directly unless you do some complex API calls combination. I think you can use: GET /v1/sources/swift-cluster-1/projects/project/meters/meter in that case, no? I actually didn't see this API you mentioned above defined anywhere. Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] How does ceilometer work with multi-DC scenario
Hi, You can deploy several ceilometer and use several databases, or just one and use a different 'source' field for each of your region/cluster to differentiate where meters come from. Thanks, Julien, I didn't know source it's meant to be presenting region/cluster info. I guess this wiki should be updated :) http://wiki.openstack.org/EfficientMetering In the field definition, source has two question marks there. Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] How does ceilometer work with multi-DC scenario
You can deploy several ceilometer and use several databases, or just one and use a different 'source' field for each of your region/cluster to differentiate where meters come from. Let's say if you deploy one ceilometer instance for two swift clusters 'swift-cluster-1' and 'swift-cluster-2'. According to the design, if we use 'source' field for region/cluster to tell what cluster the meters come from. The Ceilometer API doesn't seem being well thought through yet. A single region case: If you want to get the meters for TenantA, this API will simply do the job: GET /v1/projects/(project)/meters/(meter) Two regions case: If you want to get the meters for TenantA from 'swift-cluster-1' specifically. According to the API design we have now, there is no API can do that directly unless you do some complex API calls combination. Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] How does ceilometer work with multi-DC scenario
Hi, Has anybody come across the scenario you need to deploy two or more openstack swift or nova clusters for whatever DR or HA reasons. How Ceilometer is going to cope with that? Just wondering is there any plans or blueprints addressing the usage data replication/distinguish/isolation among multi-DCs? Thanks, Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
Hi guys, The blueprint is registered : https://blueprints.launchpad.net/ceilometer/+spec/hbase-storage-backend, Any idea what's the next step ? :) Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
Implementing? :) And eventually adding more blueprint to depends on if you can't implement this one directly because of some missing stuff. Two questions: 1. Does it need to be approved first, what's the process to get it approved/assigned to a release? Btw, I was trying to add Julien in the approver list as well, didn't manage to do that. 2. The community meeting normally is the place to have this kind of tech discussion or it's for something else? Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
Thanks Julien, I will write up a blueprint soon. Shengjie -Original Message- From: Julien Danjou [mailto:jul...@danjou.info] Sent: 07 November 2012 18:12 To: Min, Shengjie Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint? On Wed, Nov 07 2012, shengjie_...@dell.com wrote: Hi Shengjie, I am looking for a way to propose a blueprint to the ceilometer team - Adding Hbase storage backend for ceilometer. Currently we have only MongoDB and SQLalchemy as the possible ceilometer storage backend. Given the storage interface is clearly designed and allows us to have different backend, it would be nice to have more options which make Ceilometer more adoptive to some providers' existing architectures. As a cloud IAAS provider, Ceilometer might not be just considered as a usage db, also it's being seen as a centralized place to store all the usage data cross platforms. Additionally, all the historical usage data stored can be used for historical usage analysis as well, this is where MapReduce comes into play. With Hadoop Hbase, it allows usage data to be stored in HDFS, also it gives us the ability to run large scale massive MapRed analysis jobs in the future. This is indeed a good idea, and as you pointed out, Ceilometer collector has been designed so this is being possible. I think you can create a blueprint here: https://blueprints.launchpad.net/ceilometer And write in what should be done. -- Julien Danjou ;; Free Software hacker freelance ;; http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
Hi Doug, +1 It will be good to see if the storage API design works for other backends like HBase. The first glance I had on ceilometer/storage/base.py, the design of basic storage class does look ok to work with. The only challenge is that the Hbase schema is not as flexible as traditional RDBMS or MongoDB, so the efforts would be more on the Hbase implementation side. Eg. the design of row key, column family etc. We can discuss more once I had a blueprint draft registered :) -- Thanks, Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
Hi, I am looking for a way to propose a blueprint to the ceilometer team - Adding Hbase storage backend for ceilometer. Currently we have only MongoDB and SQLalchemy as the possible ceilometer storage backend. Given the storage interface is clearly designed and allows us to have different backend, it would be nice to have more options which make Ceilometer more adoptive to some providers' existing architectures. As a cloud IAAS provider, Ceilometer might not be just considered as a usage db, also it's being seen as a centralized place to store all the usage data cross platforms. Additionally, all the historical usage data stored can be used for historical usage analysis as well, this is where MapReduce comes into play. With Hadoop Hbase, it allows usage data to be stored in HDFS, also it gives us the ability to run large scale massive MapRed analysis jobs in the future. Thanks, Shengjie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
