Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-22 Thread Sean Dague
On 03/21/2014 05:11 PM, Joe Gordon wrote: On Fri, Mar 21, 2014 at 4:04 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: On 03/20/2014 06:18 PM, Joe Gordon wrote: On Thu, Mar 20, 2014 at 3:03 PM, Alexei Kornienko alexei.kornie...@gmail.com

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-21 Thread Sean Dague
On 03/20/2014 06:18 PM, Joe Gordon wrote: On Thu, Mar 20, 2014 at 3:03 PM, Alexei Kornienko alexei.kornie...@gmail.com mailto:alexei.kornie...@gmail.com wrote: Hello, We've done some profiling and results are quite interesting: during 1,5 hour ceilometer inserted 59755

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-21 Thread Boris Pavlovic
Sean, Absolutely agree with you. It's not the same to execute query and get plain text, and execute query and get hierarchy of python objects. Plus I disagree when I hear that SQLAlchemy is slow. It's slow when you are using it wrong. Like in Nova Scheduler [1] we were fetching full 3 tables

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-21 Thread Doug Hellmann
On Fri, Mar 21, 2014 at 7:04 AM, Sean Dague s...@dague.net wrote: On 03/20/2014 06:18 PM, Joe Gordon wrote: On Thu, Mar 20, 2014 at 3:03 PM, Alexei Kornienko alexei.kornie...@gmail.com mailto:alexei.kornie...@gmail.com wrote: Hello, We've done some profiling and results

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-21 Thread Joe Gordon
On Fri, Mar 21, 2014 at 4:04 AM, Sean Dague s...@dague.net wrote: On 03/20/2014 06:18 PM, Joe Gordon wrote: On Thu, Mar 20, 2014 at 3:03 PM, Alexei Kornienko alexei.kornie...@gmail.com mailto:alexei.kornie...@gmail.com wrote: Hello, We've done some profiling and results

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-21 Thread Joe Gordon
On Fri, Mar 21, 2014 at 8:58 AM, Doug Hellmann doug.hellm...@dreamhost.comwrote: On Fri, Mar 21, 2014 at 7:04 AM, Sean Dague s...@dague.net wrote: On 03/20/2014 06:18 PM, Joe Gordon wrote: On Thu, Mar 20, 2014 at 3:03 PM, Alexei Kornienko alexei.kornie...@gmail.com

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-21 Thread Alexei Kornienko
Hello, Please see some comments inline. Best Regards, Alexei Kornienko On 03/21/2014 11:11 PM, Joe Gordon wrote: On Fri, Mar 21, 2014 at 4:04 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: On 03/20/2014 06:18 PM, Joe Gordon wrote: On Thu, Mar 20, 2014

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-21 Thread Doug Hellmann
On Fri, Mar 21, 2014 at 5:13 PM, Joe Gordon joe.gord...@gmail.com wrote: On Fri, Mar 21, 2014 at 8:58 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Fri, Mar 21, 2014 at 7:04 AM, Sean Dague s...@dague.net wrote: On 03/20/2014 06:18 PM, Joe Gordon wrote: On Thu, Mar

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Nadya Privalova
Hi all, First of all, thanks for your suggestions! To summarize the discussions here: 1. We are not going to install Mongo (because is's wrong ?) 2. Idea about spawning several collectors is suspicious (btw there is a patch that run several collectors: https://review.openstack.org/#/c/79962/.)

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Sean Dague
On 03/20/2014 05:49 AM, Nadya Privalova wrote: Hi all, First of all, thanks for your suggestions! To summarize the discussions here: 1. We are not going to install Mongo (because is's wrong ?) We are not going to install Mongo not from base distribution, because we don't do that for things

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Nadya Privalova
Sean, thank for analysis. JFYI, I did some initial profiling, it's described here https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg19030.html. On Thu, Mar 20, 2014 at 2:15 PM, Sean Dague s...@dague.net wrote: On 03/20/2014 05:49 AM, Nadya Privalova wrote: Hi all, First of

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Tim Bell
March 2014 11:16 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate ... While I agree that Tempest's job is not to test performance, we do have to give some basic sanity checking

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Nadya Privalova
) Subject: Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate ... While I agree that Tempest's job is not to test performance, we do have to give some basic sanity checking here that the software is running in some performance profile that we believe

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Christian Berendt
On 03/20/2014 01:27 PM, Nadya Privalova wrote: Tim, yep. If you use one db for Ceilometer and Nova then nova's performance may be affected. If I understood it correctly the problem is not the higher load produced directly by Ceilometer on the database. The problem is that the Ceilometer compute

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Tim Bell
] Sent: 20 March 2014 11:16 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate ... While I agree that Tempest's job is not to test performance, we do have to give some basic sanity

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread David Kranz
On 03/20/2014 06:15 AM, Sean Dague wrote: On 03/20/2014 05:49 AM, Nadya Privalova wrote: Hi all, First of all, thanks for your suggestions! To summarize the discussions here: 1. We are not going to install Mongo (because is's wrong ?) We are not going to install Mongo not from base

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Sean Dague
On 03/20/2014 11:35 AM, David Kranz wrote: On 03/20/2014 06:15 AM, Sean Dague wrote: On 03/20/2014 05:49 AM, Nadya Privalova wrote: Hi all, First of all, thanks for your suggestions! To summarize the discussions here: 1. We are not going to install Mongo (because is's wrong ?) We are not

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread David Kranz
On 03/20/2014 12:31 PM, Sean Dague wrote: On 03/20/2014 11:35 AM, David Kranz wrote: On 03/20/2014 06:15 AM, Sean Dague wrote: On 03/20/2014 05:49 AM, Nadya Privalova wrote: Hi all, First of all, thanks for your suggestions! To summarize the discussions here: 1. We are not going to install

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Sean Dague
On 03/20/2014 01:01 PM, David Kranz wrote: On 03/20/2014 12:31 PM, Sean Dague wrote: On 03/20/2014 11:35 AM, David Kranz wrote: On 03/20/2014 06:15 AM, Sean Dague wrote: On 03/20/2014 05:49 AM, Nadya Privalova wrote: Hi all, First of all, thanks for your suggestions! To summarize the

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Jay Pipes
On Fri, 2014-03-21 at 00:03 +0200, Alexei Kornienko wrote: Hello, We've done some profiling and results are quite interesting: during 1,5 hour ceilometer inserted 59755 events (59755 calls to record_metering_data) this calls resulted in total 2591573 SQL queries. Yes, this matches my own

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Joe Gordon
On Thu, Mar 20, 2014 at 3:03 PM, Alexei Kornienko alexei.kornie...@gmail.com wrote: Hello, We've done some profiling and results are quite interesting: during 1,5 hour ceilometer inserted 59755 events (59755 calls to record_metering_data) this calls resulted in total 2591573 SQL queries.

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Alexei Kornienko
On 03/21/2014 12:15 AM, Jay Pipes wrote: On Fri, 2014-03-21 at 00:03 +0200, Alexei Kornienko wrote: Hello, We've done some profiling and results are quite interesting: during 1,5 hour ceilometer inserted 59755 events (59755 calls to record_metering_data) this calls resulted in total 2591573

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Jay Pipes
On Fri, 2014-03-21 at 00:32 +0200, Alexei Kornienko wrote: On 03/21/2014 12:15 AM, Jay Pipes wrote: On Fri, 2014-03-21 at 00:03 +0200, Alexei Kornienko wrote: Hello, We've done some profiling and results are quite interesting: during 1,5 hour ceilometer inserted 59755 events (59755

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Alexei Kornienko
On 03/21/2014 12:53 AM, Jay Pipes wrote: On Fri, 2014-03-21 at 00:32 +0200, Alexei Kornienko wrote: On 03/21/2014 12:15 AM, Jay Pipes wrote: On Fri, 2014-03-21 at 00:03 +0200, Alexei Kornienko wrote: Hello, We've done some profiling and results are quite interesting: during 1,5 hour

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Gordon Chung
Alexei, awesome work. Rollbacks are caused not by retry logic but by create_or_update logic: We first try to do INSERT in sub-transaction when it fails we rollback this transaction and do update instead. if you take a look at my patch addressing deadlocks(

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread Jay Pipes
On Fri, 2014-03-21 at 01:02 +0200, Alexei Kornienko wrote: On 03/21/2014 12:53 AM, Jay Pipes wrote: On Fri, 2014-03-21 at 00:32 +0200, Alexei Kornienko wrote: On 03/21/2014 12:15 AM, Jay Pipes wrote: On Fri, 2014-03-21 at 00:03 +0200, Alexei Kornienko wrote: Hello, We've done some

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-19 Thread Nadya Privalova
: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate On Tue, Mar 18, 2014 at 5:21 PM, Sean Dague s...@dague.net wrote: So I'm still -1 at the point in making UCA our default run environment until it's provably functional for a period of time. Because

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-19 Thread Joe Gordon
things running at all times) ? Tim *From:* Chmouel Boudjnah [mailto:chmo...@enovance.com] *Sent:* 18 March 2014 17:58 *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-19 Thread Doug Hellmann
The ceilometer collector is meant to scale horizontally. Have you tried configuring the test environment to run more than one copy, to process the notifications more quickly? Doug On Tue, Mar 18, 2014 at 8:09 AM, Nadya Privalova nprival...@mirantis.comwrote: Hi folks, I'd like to discuss

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-19 Thread Joe Gordon
On Wed, Mar 19, 2014 at 3:09 PM, Doug Hellmann doug.hellm...@dreamhost.comwrote: The ceilometer collector is meant to scale horizontally. Have you tried configuring the test environment to run more than one copy, to process the notifications more quickly? FYI:

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-19 Thread Sean Dague
On 03/19/2014 06:09 PM, Doug Hellmann wrote: The ceilometer collector is meant to scale horizontally. Have you tried configuring the test environment to run more than one copy, to process the notifications more quickly? The ceilometer collector is already one of the top running processes on

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-18 Thread Sean Dague
On 03/18/2014 08:09 AM, Nadya Privalova wrote: Hi folks, I'd like to discuss Ceilometer's tempest situation with you. Now we have several patch sets on review that test core functionality of Ceilometer: notificaton and pollstering (topic

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-18 Thread Julien Danjou
On Tue, Mar 18 2014, Sean Dague wrote: There is a fundamental problem here that the Ceilometer team requires a version of Mongo that's not provided by the distro. We've taken a pretty hard line on not requiring newer versions of non python stuff than the distros we support actually have.

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-18 Thread Sean Dague
On 03/18/2014 09:02 AM, Julien Danjou wrote: On Tue, Mar 18 2014, Sean Dague wrote: There is a fundamental problem here that the Ceilometer team requires a version of Mongo that's not provided by the distro. We've taken a pretty hard line on not requiring newer versions of non python stuff

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-18 Thread Julien Danjou
On Tue, Mar 18 2014, Sean Dague wrote: We've not required UCA for any other project to pass the gate. So what is the issue with Mongo 2.0.4 that makes it unsupportable in ceilometer? We require features not present in MongoDB 2.2. -- Julien Danjou -- Free Software hacker --

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-18 Thread Chmouel Boudjnah
On Tue, Mar 18, 2014 at 2:09 PM, Sean Dague s...@dague.net wrote: We've not required UCA for any other project to pass the gate. Is it that bad to have UCA in default devstack, as far as I know UCA is the official way to do OpenStack on ubuntu, right? Chmouel.

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-18 Thread Sean Dague
On 03/18/2014 12:09 PM, Chmouel Boudjnah wrote: On Tue, Mar 18, 2014 at 2:09 PM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: We've not required UCA for any other project to pass the gate. Is it that bad to have UCA in default devstack, as far as I know UCA is the

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-18 Thread Chmouel Boudjnah
On Tue, Mar 18, 2014 at 5:21 PM, Sean Dague s...@dague.net wrote: So I'm still -1 at the point in making UCA our default run environment until it's provably functional for a period of time. Because working around upstream distro breaks is no fun. I agree, if UCA is not very stable ATM, this

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-18 Thread Tim Bell
[mailto:chmo...@enovance.com] Sent: 18 March 2014 17:58 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate On Tue, Mar 18, 2014 at 5:21 PM, Sean Dague s...@dague.netmailto:s...@dague.net