Re: [openstack-dev] Which program for Rally

2014-08-11 Thread Matthew Treinish
I apologize for the delay in my response to this thread, between travelling and having a stuck 'a' key on my laptop this is the earliest I could respond. I opted for a separate branch on this thread to summarize my views and I'll respond inline later on some of the previous discussion. On Wed,

Re: [openstack-dev] Which program for Rally

2014-08-12 Thread Matthew Treinish
On Mon, Aug 11, 2014 at 07:06:11PM -0400, Zane Bitter wrote: On 11/08/14 16:21, Matthew Treinish wrote: I'm sorry, but the fact that the docs in the rally tree has a section for user testimonials [4] I feel speaks a lot about the intent of the project. What... does that even mean? Yeah

[openstack-dev] [QA] Meeting Thursday August 14th at 17:00 UTC

2014-08-13 Thread Matthew Treinish
Hi Everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be this Thursday, August 14th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add

Re: [openstack-dev] Which program for Rally

2014-08-13 Thread Matthew Treinish
://lists.openstack.org/pipermail/openstack-dev/2013-October/017004.html [2] http://lists.openstack.org/pipermail/openstack-dev/2014-August/042269.html On Tue, Aug 12, 2014 at 12:37 AM, David Kranz dkr...@redhat.com wrote: On 08/11/2014 04:21 PM, Matthew Treinish wrote: I apologize

Re: [openstack-dev] [all] The future of the integrated release

2014-08-13 Thread Matthew Treinish
On Wed, Aug 13, 2014 at 03:43:21PM -0400, Eoghan Glynn wrote: Divert all cross project efforts from the following projects so we can focus our cross project resources. Once we are in a bitter place we can expand our cross project resources to cover these again. This doesn't mean

[openstack-dev] [QA] Picking a Name for the Tempest Library

2014-08-15 Thread Matthew Treinish
Hi Everyone, So as part of splitting out common functionality from tempest into a library [1] we need to create a new repository. Which means we have the fun task of coming up with something to name it. I'm personally thought we should call it: - mesocyclone Which has the advantage of being a

Re: [openstack-dev] Which program for Rally

2014-08-18 Thread Matthew Treinish
On Fri, Aug 15, 2014 at 01:57:29AM +0400, Boris Pavlovic wrote: Matt, One thing did just occur to me while writing this though it's probably worth investigating splitting out the stress test framework as an external tool/project after we start work on the tempest library. [3] I

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library

2014-08-18 Thread Matthew Treinish
On Sat, Aug 16, 2014 at 06:27:19PM +0200, Marc Koderer wrote: Hi all, Am 15.08.2014 um 23:31 schrieb Jay Pipes jaypi...@gmail.com: I suggest that tempest should be the name of the import'able library, and that the integration tests themselves should be what is pulled out of the

[openstack-dev] [QA] Meeting Thursday August 21st at 22:00 UTC

2014-08-20 Thread Matthew Treinish
Hi everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, August 21st at 22:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library

2014-08-22 Thread Matthew Treinish
On Fri, Aug 15, 2014 at 03:14:21PM -0400, Matthew Treinish wrote: Hi Everyone, So as part of splitting out common functionality from tempest into a library [1] we need to create a new repository. Which means we have the fun task of coming up with something to name it. I'm personally

Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-26 Thread Matthew Treinish
On Tue, Aug 26, 2014 at 10:04:41AM -0400, Doug Hellmann wrote: On Aug 26, 2014, at 5:13 AM, Thierry Carrez thie...@openstack.org wrote: OK, now that we have evacuated the terminology issue (we'll use liaison or janitor or secretary, not czar), and side-stepped the offtopic development

[openstack-dev] [QA] Meeting Thursday August 28th at 17:00 UTC

2014-08-27 Thread Matthew Treinish
Hi Everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be this Thursday, August 28th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add

Re: [openstack-dev] [all] gate debugging

2014-08-27 Thread Matthew Treinish
On Wed, Aug 27, 2014 at 05:47:09PM -0400, Doug Hellmann wrote: On Aug 27, 2014, at 5:27 PM, Doug Hellmann d...@doughellmann.com wrote: On Aug 27, 2014, at 2:54 PM, Sean Dague s...@dague.net wrote: Note: thread intentionally broken, this is really a different topic. On

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library

2014-08-28 Thread Matthew Treinish
On Fri, Aug 22, 2014 at 11:26:25AM -0400, Matthew Treinish wrote: On Fri, Aug 15, 2014 at 03:14:21PM -0400, Matthew Treinish wrote: Hi Everyone, So as part of splitting out common functionality from tempest into a library [1] we need to create a new repository. Which means we have

Re: [openstack-dev] [qa][all][Heat] Packaging of functional tests

2014-09-04 Thread Matthew Treinish
On Thu, Sep 04, 2014 at 04:32:53PM +0100, Steven Hardy wrote: On Thu, Sep 04, 2014 at 10:45:59AM -0400, Jay Pipes wrote: On 08/29/2014 05:15 PM, Zane Bitter wrote: On 29/08/14 14:27, Jay Pipes wrote: On 08/26/2014 10:14 AM, Zane Bitter wrote: Steve Baker has started the process of moving

Re: [openstack-dev] [qa][all][Heat] Packaging of functional tests

2014-09-05 Thread Matthew Treinish
On Fri, Sep 05, 2014 at 09:42:17AM +1200, Steve Baker wrote: On 05/09/14 04:51, Matthew Treinish wrote: On Thu, Sep 04, 2014 at 04:32:53PM +0100, Steven Hardy wrote: On Thu, Sep 04, 2014 at 10:45:59AM -0400, Jay Pipes wrote: On 08/29/2014 05:15 PM, Zane Bitter wrote: On 29/08/14 14:27

[openstack-dev] [QA] Meeting Thursday September 11th at 17:00 UTC

2014-09-10 Thread Matthew Treinish
Hi Everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be this Thursday, September 11th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to

[openstack-dev] [QA] Meeting Thursday September 18th at 22:00 UTC

2014-09-17 Thread Matthew Treinish
Hi everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, September 18th at 22:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome

[openstack-dev] [QA] PTL Candidacy

2014-09-22 Thread Matthew Treinish
I'd like to throw my hat into the ring for another term as PTL of the QA program. The Juno cycle has been very productive with some big developments in the QA program including: being the first cycle to implement branchless Tempest, starting tempest-lib, and consolidating the devstack program

[openstack-dev] [QA] Meeting Thursday September 25th at 17:00 UTC

2014-09-24 Thread Matthew Treinish
Hi Everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be this Thursday, September 25th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to

Re: [openstack-dev] [QA] Tempest blueprints status update and rationale, input demanded

2013-12-11 Thread Matthew Treinish
On Wed, Dec 11, 2013 at 01:44:19PM +0100, Giulio Fidente wrote: hi, I'm attempting to rationalize on the status of tempest blueprints. I need your help so I organized questions in a few open points. * (1) I'm looking for input here on the actual status of the following blueprints, which

[openstack-dev] [QA] Meeting Thursday December 12th at 22:00 UTC

2013-12-11 Thread Matthew Treinish
The weekly OpenStack QA team IRC meeting will be tomorrow, December 12th at 22:00 UTC in the #openstack-meeting channel. The meeting agenda can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to the agenda. Also, a quick reminder that

[openstack-dev] [QA] Meeting Thursday December 19th at 17:00 UTC

2013-12-18 Thread Matthew Treinish
Just a reminder that the weekly OpenStack QA team IRC meeting will be tomorrow, December 19th at 17:00 UTC in the #openstack-meeting channel. The meeting agenda can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to the agenda. -Matt

[openstack-dev] [QA] Meeting Thursday January 9th at 22:00 UTC

2014-01-06 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be this Thursday, January 9th at 22:00 UTC in the #openstack-meeting channel. I'm sending this reminder out a bit earlier than usual this week because of the 2 week break over the holidays. The meeting agenda can be found

[openstack-dev] [QA] Changes to Tempest run_tests.sh

2014-01-11 Thread Matthew Treinish
Hi everyone, I just wanted to bring up some changes that recently merged to tempest. As part of the tempest unit tests blueprint I converted the run_tests.sh script to execute unit tests instead of running tempest itself. This makes the run_tests.sh script consistent with the other projects to

Re: [openstack-dev] [nova] api schema validation pattern changes

2014-01-12 Thread Matthew Treinish
snip Last, is a question, is it possible to currently run the full API an build a json schema for it all? Or to query these validating schemas? We *really* want that over in tempest, so we can completely drop the manual creation of negative testing, and just fuzz known bad against the

Re: [openstack-dev] [tempest] testr duplicate test id detected

2014-01-12 Thread Matthew Treinish
On Mon, Jan 13, 2014 at 08:49:35AM +0530, Bhuvan Arumugam wrote: I'm facing this issue when listing tests using testr. Anyone else facing this issue? [bhuvan@build tempest]$ tools/with_venv.sh testr list-tests running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \

[openstack-dev] [QA] Meeting Thursday January 16th at 17:00 UTC

2014-01-15 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tommorrow Thursday, January 16th at 17:00 UTC in the #openstack-meeting channel. The meeting agenda can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to the agenda.

[openstack-dev] [QA] The future of nosetests with Tempest

2014-01-16 Thread Matthew Treinish
Hi everyone, With some recent changes made to Tempest compatibility with nosetests is going away. We've started using newer features that nose just doesn't support. One example of this is that we've started using testscenarios and we're planning to do this in more places moving forward. So at

Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-16 Thread Matthew Treinish
On Wed, Jan 15, 2014 at 11:20:22AM -0500, Yair Fried wrote: Hi Guys As Maru pointed out - NetworkBasicOps scenario has grown out of proportion and is no longer basic ops. Is your issue here that it's just called basic ops and you don't think that's reflective of what is being tested in that

Re: [openstack-dev] [QA] The future of nosetests with Tempest

2014-01-17 Thread Matthew Treinish
On Fri, Jan 17, 2014 at 08:32:19AM -0500, David Kranz wrote: On 01/16/2014 10:56 PM, Matthew Treinish wrote: Hi everyone, With some recent changes made to Tempest compatibility with nosetests is going away. We've started using newer features that nose just doesn't support. One example

[openstack-dev] [QA] Meeting Thursday January 23rd at 22:00UTC

2014-01-22 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tommorrow Thursday, January 23rd at 22:00 UTC in the #openstack-meeting channel. The agenda for tommorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an

[openstack-dev] [QA] Meeting Thursday January 30th at 17:00UTC

2014-01-29 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, January 30th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item

[openstack-dev] [QA] Meeting Thursday February 6th at 22:00UTC

2014-02-05 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, February 6th at 22:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item

Re: [openstack-dev] [Neutron][QA] Making tests symmetric again

2014-02-07 Thread Matthew Treinish
On Fri, Feb 07, 2014 at 03:58:28PM +0100, Salvatore Orlando wrote: I just spoke with Eugene over IRC and the problem is that the test attempts to connect to an instance over a private network, which cannot work in isolated mode. This patch will skip the load balancing scenario:

Re: [openstack-dev] [QA] The future of nosetests with Tempest

2014-02-12 Thread Matthew Treinish
On Wed, Feb 12, 2014 at 11:32:39AM -0700, Matt Riedemann wrote: On 1/17/2014 8:34 AM, Matthew Treinish wrote: On Fri, Jan 17, 2014 at 08:32:19AM -0500, David Kranz wrote: On 01/16/2014 10:56 PM, Matthew Treinish wrote: Hi everyone, With some recent changes made to Tempest compatibility

[openstack-dev] [QA] Meeting Thursday February 13th at 17:00UTC

2014-02-12 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, February 13th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item

Re: [openstack-dev] [All] Fixed recent gate issues

2014-02-18 Thread Matthew Treinish
On Tue, Feb 18, 2014 at 08:27:16PM +0100, Alan Pevec wrote: Hi John, thanks for the summary. I've noticed one more fall out from swiftclient update in Grenade jobs running on stable/havana changes e.g. http://logs.openstack.org/02/73402/1/check/check-grenade-dsvm/a5650ac/console.html ...

[openstack-dev] [QA] Meeting Thursday February 20th at 22:00UTC

2014-02-19 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, February 20th at 22:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item

[openstack-dev] [QA] Meeting Thursday February 27th at 17:00UTC

2014-02-26 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, February 27th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item

Re: [openstack-dev] [QA] The future of nosetests with Tempest

2014-02-27 Thread Matthew Treinish
On Tue, Feb 25, 2014 at 07:46:23PM -0600, Matt Riedemann wrote: On 2/12/2014 1:57 PM, Matthew Treinish wrote: On Wed, Feb 12, 2014 at 11:32:39AM -0700, Matt Riedemann wrote: On 1/17/2014 8:34 AM, Matthew Treinish wrote: On Fri, Jan 17, 2014 at 08:32:19AM -0500, David Kranz wrote

[openstack-dev] [QA] Meeting Thursday March 6th at 22:00UTC

2014-03-05 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, March 6th at 22:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to

[openstack-dev] [QA] Meeting Thursday March 13th at 17:00UTC

2014-03-12 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, March 13th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to

[openstack-dev] [Ceilometer] [QA] Slow Ceilometer resource_list CLI command

2014-03-17 Thread Matthew Treinish
Hi everyone, So a little while ago we noticed that in all the gate runs one of the ceilometer cli tests is consistently in the list of slowest tests. (and often the slowest) This was a bit surprising given the nature of the cli tests we expect them to execute very quickly.

Re: [openstack-dev] [qa] API schema location

2014-03-18 Thread Matthew Treinish
On Tue, Mar 18, 2014 at 12:34:57PM +0100, Koderer, Marc wrote: On Tue, 18 Marc 2014 12:00:00 +0100 Christopher Yeoh [mailto:cbky...@gmail.com] wrote: On Tue, 18 Mar 2014 10:39:19 +0100 Koderer, Marc m.kode...@telekom.de wrote: I just recognized that we have very similar interface

Re: [openstack-dev] [Openstack-dev] [Nova] use Keystone V3 token to volume attachment

2014-03-19 Thread Matthew Treinish
On Wed, Mar 19, 2014 at 09:35:34AM -0500, Matt Riedemann wrote: On 3/19/2014 2:48 AM, Shao Kai SK Li wrote: Hello: I am working on this patch(https://review.openstack.org/#/c/77524/) to fix bugs about volume attach failure with keystone V3 token. Just wonder, is there

[openstack-dev] [QA] Meeting Thursday March 20th at 22:00UTC

2014-03-19 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, March 20th at 22:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-20 Thread Matthew Treinish
On Thu, Mar 20, 2014 at 11:35:15AM +, Malini Kamalambal wrote: Hello all, I have been working on adding tests in Tempest for Marconi, for the last few months. While there are many amazing people to work with, the process has been more difficult than I expected. Couple of

Re: [openstack-dev] Spec repos for blueprint development and review

2014-03-24 Thread Matthew Treinish
On Mon, Mar 24, 2014 at 02:32:35PM -0400, Russell Bryant wrote: On 03/24/2014 02:19 PM, Monty Taylor wrote: On 03/24/2014 10:07 AM, Russell Bryant wrote: On 03/24/2014 12:34 PM, James E. Blair wrote: Hi, So recently we started this experiment with the compute and qa programs to try

Re: [openstack-dev] [qa] [neutron] Neutron Full Parallel job - Last 4 days failures

2014-03-24 Thread Matthew Treinish
On Mon, Mar 24, 2014 at 09:56:09PM +0100, Salvatore Orlando wrote: Thanks a lot! We now need to get on these bugs, and define with QA an acceptable failure rate criterion for switching the full job to voting. It would be good to have a chance to only run the tests against code which is

[openstack-dev] [QA] Meeting Thursday March 27th at 17:00UTC

2014-03-26 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, March 27th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to

[openstack-dev] QA Program PTL Candidacy

2014-03-28 Thread Matthew Treinish
I would like to run for the QA Program PTL position. I'm currently a core reviewer on the Tempest and elastic-recheck projects and a member of the stable maint team. I have been working on Tempest and on improving overall OpenStack quality since I started contributing to the project late in the

[openstack-dev] [QA] Meeting Thursday April 3rd at 22:00UTC

2014-04-02 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, April 3rd at 22:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to

Re: [openstack-dev] [nova][powervm] my notes from the meeting on powervm CI

2013-10-10 Thread Matthew Treinish
On Thu, Oct 10, 2013 at 04:55:51PM -0500, Matt Riedemann wrote: Based on the discussion with Russell and Dan Smith in the nova meeting today, here are some of my notes from the meeting that can continue the discussion. These are all pretty rough at the moment so please bear with me, this

Re: [openstack-dev] [nova][powervm] my notes from the meeting on powervm CI

2013-10-10 Thread Matthew Treinish
On Thu, Oct 10, 2013 at 07:39:37PM -0700, Joe Gordon wrote: On Thu, Oct 10, 2013 at 7:28 PM, Matt Riedemann mrie...@us.ibm.com wrote: 4. What is the max amount of time for us to report test results? Dan didn't seem to think 48 hours would fly. :) Honestly, I think that 12 hours

Re: [openstack-dev] [qa] Update: Nova API List for Missing Tempest Tests

2013-10-15 Thread Matthew Treinish
On Tue, Oct 15, 2013 at 06:25:28AM +, Masayuki Igawa wrote: Hi, First, thank you to an anonymous for updating this list! - GET /{project_id}/servers/:server_id/diagnostics And, I have updated: Nova API List for Missing Tempest Tests.

Re: [openstack-dev] [Tempest][Neutron] Discovering enabled extensions

2013-10-17 Thread Matthew Treinish
On Thu, Oct 17, 2013 at 10:10:15AM -0400, Jay Pipes wrote: On 10/17/2013 08:46 AM, Salvatore Orlando wrote: Hi, in the discussion for patch https://review.openstack.org/#/c/50880/ Sean asked a very reasonable question: so are all these [Neutron] extensions always loaded on all

Re: [openstack-dev] [Tempest][Neutron] Discovering enabled extensions

2013-10-17 Thread Matthew Treinish
in the commit I pointed out below. On 17 October 2013 17:34, Jay Pipes jaypi...@gmail.com wrote: On 10/17/2013 10:46 AM, Matthew Treinish wrote: On Thu, Oct 17, 2013 at 10:10:15AM -0400, Jay Pipes wrote: On 10/17/2013 08:46 AM, Salvatore Orlando wrote: Hi, in the discussion for patch

Re: [openstack-dev] [qa] Proposals for Tempest core

2013-11-16 Thread Matthew Treinish
I think responding to this is worth breaking my vacation email embargo. Although I seem to remember a certain PTL whose vacation we waited for the last time we did core proposals... Sean Dague s...@dague.net wrote: It's post summit time, so time to evaluate our current core group for Tempest.

Re: [openstack-dev] [qa] Plan for failing successful tempest jobs when new ERRORs appear in logs

2013-11-25 Thread Matthew Treinish
On Mon, Nov 25, 2013 at 11:06:36AM -0800, Joe Gordon wrote: On Mon, Nov 18, 2013 at 2:58 PM, David Kranz dkr...@redhat.com wrote: So we are close to being able to start doing this. The current whitelist is here https://github.com/openstack/tempest/blob/master/etc/ whitelist.yaml. I have a

Re: [openstack-dev] [qa] Major change to tempest.conf.sample coming

2013-11-27 Thread Matthew Treinish
On Wed, Nov 13, 2013 at 02:42:34PM -0500, David Kranz wrote: This is a heads up that soon we will be auto-generating the tempest.conf.sample from the tempest code that uses oslo.config. Following in the footsteps of nova, this should reduce bugs around failures to keep the config code and the

[openstack-dev] [qa] Moving the QA meeting time

2013-12-04 Thread Matthew Treinish
Hi everyone, I'm looking at changing our weekly QA meeting time to make it more globally attendable. Right now the current time of 17:00 UTC doesn't really work for people who live in Asia Pacific timezones. (which includes a third of the current core review team) There are 2 approaches that I

Re: [openstack-dev] [qa] Moving the QA meeting time

2013-12-05 Thread Matthew Treinish
On Wed, Dec 04, 2013 at 04:04:22PM -0500, Matthew Treinish wrote: Hi everyone, I'm looking at changing our weekly QA meeting time to make it more globally attendable. Right now the current time of 17:00 UTC doesn't really work for people who live in Asia Pacific timezones. (which includes

Re: [openstack-dev] [qa] Should Assignee, Milestone really be in the qa-spec?

2014-06-11 Thread Matthew Treinish
On Wed, Jun 11, 2014 at 03:17:48PM -0400, David Kranz wrote: While reviewing some specs I noticed that I had put myself down for more Juno-2 work than is likely to be completed. I suspect this will happen routinely with many folks. Also, assignees may change. This information is not really

[openstack-dev] [Nova][QA] Disabling the v3 API tests in the gate

2014-06-11 Thread Matthew Treinish
Hi everyone, As part of debugging all the bugs that have been plaguing the gate the past couple of weeks one of the things that came up is that we're still running the v3 API tests in the gate. AIUI at summit Nova decided that the v3 API test won't exist as a separate major version. So I'm not

Re: [openstack-dev] [Nova][QA] Disabling the v3 API tests in the gate

2014-06-12 Thread Matthew Treinish
On Fri, Jun 13, 2014 at 12:41:19AM +0930, Christopher Yeoh wrote: On Fri, Jun 13, 2014 at 12:25 AM, Dan Smith d...@danplanet.com wrote: I think it'd be OK to move them to the experimental queue and a periodic nightly job until the v2.1 stuff shakes out. The v3 API is marked experimental

Re: [openstack-dev] [qa] Clarification of policy for qa-specs around adding new tests

2014-06-16 Thread Matthew Treinish
On Mon, Jun 16, 2014 at 10:46:51AM -0400, David Kranz wrote: I have been reviewing some of these specs and sense a lack of clarity around what is expected. In the pre-qa-specs world we did not want tempest blueprints to be used by projects to track their tempest test submissions because the

Re: [openstack-dev] [Nova][QA] Disabling the v3 API tests in the gate

2014-06-17 Thread Matthew Treinish
On Thu, Jun 12, 2014 at 12:10:23PM -0400, Sean Dague wrote: On 06/12/2014 12:02 PM, Matt Riedemann wrote: On 6/12/2014 10:51 AM, Matthew Treinish wrote: On Fri, Jun 13, 2014 at 12:41:19AM +0930, Christopher Yeoh wrote: On Fri, Jun 13, 2014 at 12:25 AM, Dan Smith d...@danplanet.com

[openstack-dev] [QA] Meeting Thursday June 19th at 17:00 UTC

2014-06-18 Thread Matthew Treinish
Hi Everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be this Thursday, June 19th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add

Re: [openstack-dev] [qa] Clarification of policy for qa-specs around adding new tests

2014-06-18 Thread Matthew Treinish
On Tue, Jun 17, 2014 at 01:45:55AM +, Kenichi Oomichi wrote: -Original Message- From: Matthew Treinish [mailto:mtrein...@kortar.org] Sent: Monday, June 16, 2014 11:58 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [qa

[openstack-dev] [QA] Meeting Thursday June 26th at 22:00 UTC

2014-06-25 Thread Matthew Treinish
Hi everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, June 26th at 22:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to

Re: [openstack-dev] [qa] issues adding functionality to javelin2

2014-07-01 Thread Matthew Treinish
On Tue, Jul 01, 2014 at 02:01:19PM +0100, Chris Dent wrote: I've been working to add ceilometer checks in javelin2. Doing so has revealed some issues that appear to be a fairly big deal but I suppose there's some chance I'm doing things completely wrong. For reference my experiments are

[openstack-dev] [QA] Spec Review Day July 9th

2014-07-01 Thread Matthew Treinish
Hi Everyone, During the last qa meeting we were discussing ways to try an increase throughput on the qa-specs repo. We decided to have a dedicated review day for the specs repo. Right now specs approval is a relatively slow process, reviews seem to take too long (an average wait time of ~12 days)

Re: [openstack-dev] [nova] how to unit test scripts outside of nova/nova?

2014-07-01 Thread Matthew Treinish
On Tue, Jul 01, 2014 at 03:21:06PM -0500, Matt Riedemann wrote: As part of the enforce-unique-instance-uuid-in-db blueprint [1] I'm writing a script to scan the database and find any NULL instance_uuid records that will cause the new database migration to fail so that operators can run this

[openstack-dev] [QA] Meeting Thursday July 3rd at 17:00 UTC

2014-07-02 Thread Matthew Treinish
Hi Everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be this Thursday, July 3rd at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an

Re: [openstack-dev] [QA] Spec Review Day July 9th

2014-07-08 Thread Matthew Treinish
On Tue, Jul 01, 2014 at 01:30:53PM -0400, Matthew Treinish wrote: Hi Everyone, During the last qa meeting we were discussing ways to try an increase throughput on the qa-specs repo. We decided to have a dedicated review day for the specs repo. Right now specs approval is a relatively

Re: [openstack-dev] [qa][all] Branchless Tempest beyond pure-API tests, impact on backporting policy

2014-07-09 Thread Matthew Treinish
On Wed, Jul 09, 2014 at 05:41:10AM -0400, Eoghan Glynn wrote: TL;DR: branchless Tempest shouldn't impact on backporting policy, yet makes it difficult to test new features not discoverable via APIs Folks, At the project/release status meeting yesterday[1], I raised the issue that

[openstack-dev] [QA] Meeting Thursday July 10th at 22:00 UTC

2014-07-09 Thread Matthew Treinish
Hi everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, July 10th at 22:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to

Re: [openstack-dev] [nova][cinder][ironic][qa] Tempest+Ironic volume attachment tests

2014-07-09 Thread Matthew Treinish
On Wed, Jul 09, 2014 at 10:58:27AM -0400, David Shrewsbury wrote: Hi! The ironic team is working on enabling the API tempest tests with ironic. However, ironic doesn’t yet completely support volume attachment, so the cinder api tests fail miserably (where attachment is needed).

Re: [openstack-dev] [qa][all] Branchless Tempest beyond pure-API tests, impact on backporting policy

2014-07-09 Thread Matthew Treinish
On Wed, Jul 09, 2014 at 01:44:33PM -0400, Eoghan Glynn wrote: Thanks for the response Matt, some comments inline. At the project/release status meeting yesterday[1], I raised the issue that featureful backports to stable are beginning to show up[2], purely to facilitate branchless

Re: [openstack-dev] [qa][all] Branchless Tempest beyond pure-API tests, impact on backporting policy

2014-07-10 Thread Matthew Treinish
On Thu, Jul 10, 2014 at 08:37:40AM -0400, Eoghan Glynn wrote: Note that the notifications that capture these resource state transitions are a long-standing mechanism in openstack that ceilometer has depended on from the very outset. I don't think it's realistic to envisage these

[openstack-dev] [QA] No Meeting this Week

2014-07-16 Thread Matthew Treinish
Hi Everyone, Just wanted to send a reminder to the list that we're not having a meeting this week. Since most people are here in Darmstadt this week at the mid-cycle meet-up there isn't a reason to have the regular weekly meeting. So I'm cancelling this week's meeting. We will have our next

[openstack-dev] [QA] Proposed Changes to Tempest Core

2014-07-21 Thread Matthew Treinish
Hi Everyone, I would like to propose 2 changes to the Tempest core team: First, I'd like to nominate Andrea Fritolli to the Tempest core team. Over the past cycle Andrea has been steadily become more actively engaged in the Tempest community. Besides his code contributions around refactoring

Re: [openstack-dev] [Neutron][QA] Enabling full neutron Job

2014-07-23 Thread Matthew Treinish
] https://launchpad.net/bugs/1329546 [2] https://launchpad.net/bugs/1333654 [3] https://review.openstack.org/#/c/99182/ [4] https://review.openstack.org/#/c/103865/ [5] https://bugs.launchpad.net/neutron/+bug/1273386 On 25 June 2014 23:38, Matthew Treinish mtrein

[openstack-dev] [QA] Meeting Thursday July 24th at 22:00 UTC

2014-07-23 Thread Matthew Treinish
Hi everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, July 24th at 22:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to

Re: [openstack-dev] [qa] Test Ceilometer polling in tempest

2014-07-24 Thread Matthew Treinish
On Wed, Jul 16, 2014 at 07:44:38PM +0400, Dina Belova wrote: Ildiko, thanks for starting this discussion. Really, that is quite painful problem for Ceilometer and QA team. As far as I know, currently there is some kind of tendency of making integration Tempest tests quicker and less resource

Re: [openstack-dev] Thoughts on the patch test failure rate and moving forward

2014-07-24 Thread Matthew Treinish
On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote: OpenStack has a substantial CI system that is core to its development process. The goals of the system are to facilitate merging good code, prevent regressions, and ensure that there is at least one configuration of upstream

Re: [openstack-dev] [QA] Proposed Changes to Tempest Core

2014-07-25 Thread Matthew Treinish
So all of the current core team members have voted unanimously in favor of adding Andrea to the team. Welcome to the team Andrea. -Matt Treinish On Fri, Jul 25, 2014 at 01:32:27PM -0400, Attila Fazekas wrote: +1 - Original Message - From: Matthew Treinish mtrein...@kortar.org

Re: [openstack-dev] Thoughts on the patch test failure rate and moving forward

2014-07-25 Thread Matthew Treinish
On Thu, Jul 24, 2014 at 06:54:38PM -0400, Sean Dague wrote: On 07/24/2014 05:57 PM, Matthew Treinish wrote: On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote: OpenStack has a substantial CI system that is core to its development process. The goals of the system

Re: [openstack-dev] DevStack program change

2014-08-03 Thread Matthew Treinish
On Fri, Aug 01, 2014 at 03:50:53PM -0500, Anne Gentle wrote: On Fri, Aug 1, 2014 at 10:48 AM, Dean Troyer dtro...@gmail.com wrote: I propose we de-program DevStack and consolidate it into the QA program. Some of my concerns about doing this in the beginning have proven to be a non-issue

[openstack-dev] [QA] Meeting Thursday April 10th at 17:00UTC

2014-04-09 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, April 10th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to

[openstack-dev] [QA] Summit Session Proposals

2014-04-14 Thread Matthew Treinish
Hi Everyone, Just wanted to send out a quick reminder that the summit session proposal submission deadline of April 20th is approaching quickly. So if you have a topic you would like to submit a session for the QA design summit track please submit them on: summit.openstack.org before the 20th.

Re: [openstack-dev] [Tempest][qa] : backporting Test Security Groups Basic Ops in stable/havana ?

2014-04-14 Thread Matthew Treinish
On Mon, Apr 14, 2014 at 07:48:57AM -0400, Yair Fried wrote: Hi Julien I've tried to backport it in the past, and it was rejected, as it needs quite a lot of work. The specific case you referenced I -2'd because of the all the required changes just to make that test work were far too large and

[openstack-dev] [QA] Meeting Thursday April 17th at 22:00UTC

2014-04-16 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, April 17th at 22:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to

[openstack-dev] [QA] qa-specs Repo and QA Program Juno Blueprint Review Process

2014-04-18 Thread Matthew Treinish
Hi Everyone, Just like Nova [1] the QA program has adopted the proposal [2] to use gerrit to review blueprint specifications. The openstack/qa-specs repo is now ready for submissions. Changes are submitted to it like any other gerrit project. The README and a template for submitting new

Re: [openstack-dev] [QA] qa-specs Repo and QA Program Juno Blueprint Review Process

2014-04-20 Thread Matthew Treinish
On Mon, Apr 21, 2014 at 01:17:37AM +, Kenichi Oomichi wrote: Hi Matthew, Thanks for doing this, -Original Message- From: Matthew Treinish [mailto:mtrein...@kortar.org] Sent: Saturday, April 19, 2014 11:59 AM To: openstack-dev@lists.openstack.org Subject: [openstack

Re: [openstack-dev] [qa] Selecting Compute tests by driver/hypervisor

2014-04-21 Thread Matthew Treinish
On Mon, Apr 21, 2014 at 10:52:39PM +, Daryl Walleck wrote: I nearly opened a spec for this, but I'd really like to get some feedback first. One of the challenges I've seen lately for Nova teams not using KVM or Xen (Ironic and LXC are just a few) is how to properly run the subset of

[openstack-dev] [QA] Meeting Thursday April 24th at 17:00UTC

2014-04-23 Thread Matthew Treinish
Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, April 24th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to

Re: [openstack-dev] [all] Branchless Tempest QA Spec - final draft

2014-04-27 Thread Matthew Treinish
On Mon, Apr 28, 2014 at 01:01:00AM +, Kenichi Oomichi wrote: Hi, Sorry for my late response, but I'd like to discuss this again. Now we are working for adding Nova API responses checks to Tempest[1] to block backward incompatible changes. With this work, Tempest checks each

[openstack-dev] [QA] No Meeting this Week

2014-04-29 Thread Matthew Treinish
Hi Everyone, Since this is the scheduled off week, I'm going to cancel this week's QA meeting. The next meeting will be on May 8th and will be at 22:00 UTC instead of having two consecutive meetings at the same time. I will send out the usual reminder email next week. Thanks, Matt Treinish

  1   2   3   4   >