Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-26 Thread Daniele Venzano
On 09/25/14 17:55, Clint Byrum wrote: Now I use Ceilometer's pipeline to forward events to elasticsearch via udp + logstash and do not use Ceilometer's DB or API at all. Interesting, this almost sounds like what should be the default configuration honestly. Ceilometer generates a lot of data

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Igor Degtiarov
Hi, Qiming Teng. Now all backends support events. So you may use MongoDB instead of MySQL, or if you like you may choose HBase. Cheers, Igor. -- Igor On Thu, Sep 25, 2014 at 7:43 AM, Preston L. Bannister pres...@bannister.us wrote: Sorry, I am jumping into this without enough context, but ...

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Qiming Teng
So MongoDB support to events is ready in tree? Regards, Qiming On Thu, Sep 25, 2014 at 10:26:08AM +0300, Igor Degtiarov wrote: Hi, Qiming Teng. Now all backends support events. So you may use MongoDB instead of MySQL, or if you like you may choose HBase. Cheers, Igor. -- Igor

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Dina Belova
Qiming, yes - for MongoDB, DB2, HBase and SQL-based, all the backends support events feature for now, this has been merged afair ~month or two ago. Cheers Dina On Thu, Sep 25, 2014 at 11:45 AM, Qiming Teng teng...@linux.vnet.ibm.com wrote: So MongoDB support to events is ready in tree?

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Qiming Teng
On Wed, Sep 24, 2014 at 09:43:54PM -0700, Preston L. Bannister wrote: Sorry, I am jumping into this without enough context, but ... On Wed, Sep 24, 2014 at 8:37 PM, Qiming Teng teng...@linux.vnet.ibm.com wrote: mysql select count(*) from metadata_text; +--+ | count(*) |

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Daniele Venzano
On 09/25/14 10:12, Qiming Teng wrote: Yes, just about 3 VMs running on two hosts, for at most 3 weeks. This is leading me to another question -- any best practices/tools to retire the old data on a regular basis? Regards, Qiming There is a tool: ceilometer-expirer I tried to use it on a

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Qiming Teng
On Thu, Sep 25, 2014 at 11:40:11AM +0200, Daniele Venzano wrote: On 09/25/14 10:12, Qiming Teng wrote: Yes, just about 3 VMs running on two hosts, for at most 3 weeks. This is leading me to another question -- any best practices/tools to retire the old data on a regular basis? Regards, Qiming

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread gordon chung
mysql select count(*) from metadata_text; +--+ | count(*) | +--+ | 25249913 | +--+ 1 row in set (3.83 sec) There were 25M records in one table. The deletion time is reaching an unacceptable level (7 minutes for 4M records) and it was not increasing in a linear

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Clint Byrum
Excerpts from Daniele Venzano's message of 2014-09-25 02:40:11 -0700: On 09/25/14 10:12, Qiming Teng wrote: Yes, just about 3 VMs running on two hosts, for at most 3 weeks. This is leading me to another question -- any best practices/tools to retire the old data on a regular basis?

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Qiming Teng
On Thu, Sep 25, 2014 at 11:51:23AM -0400, gordon chung wrote: mysql select count(*) from metadata_text; +--+ | count(*) | +--+ | 25249913 | +--+ 1 row in set (3.83 sec) There were 25M records in one table. The deletion time is reaching an unacceptable

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-24 Thread Mark Kirkwood
On 25/09/14 15:37, Qiming Teng wrote: Hi, Some weeks ago, I checked my then latest devstack install and I learned this: event support in Ceilometer is only available for sqlalchemy backend; mongodb backend was still under development. I have been using MySQL during the past weeks and now I

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-24 Thread Clint Byrum
Excerpts from Qiming Teng's message of 2014-09-24 20:37:39 -0700: Hi, Some weeks ago, I checked my then latest devstack install and I learned this: event support in Ceilometer is only available for sqlalchemy backend; mongodb backend was still under development. I have been using MySQL

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-24 Thread Preston L. Bannister
Sorry, I am jumping into this without enough context, but ... On Wed, Sep 24, 2014 at 8:37 PM, Qiming Teng teng...@linux.vnet.ibm.com wrote: mysql select count(*) from metadata_text; +--+ | count(*) | +--+ | 25249913 | +--+ 1 row in set (3.83 sec) There are