[openstack-dev] [qa] identity v3 issue causing non-admin job to fail

2015-07-14 Thread David Kranz
Now that the tempest periodic jobs are back (thanks infra!), I was looking into the real failures. It seems the main one is caused by the fact that the v3 check for primary creds fails if 'admin_domain_name' in the identity section is None, which it is when devstack configures tempest for

Re: [openstack-dev] [qa] Tempest bug triage questions

2015-06-22 Thread David Kranz
On 06/22/2015 10:15 AM, Yaroslav Lobankov wrote: Hello everyone, I have some questions about the bug triage procedure for Tempest: 1. Some bugs in Tempest have status Fix committed. Should we move statuses of these bugs to Fix released? Yes, tempest doesn't have the kind of releases where Fix

Re: [openstack-dev] [QA][Tempest] Proposing Jordan Pittier for Tempest Core

2015-06-22 Thread David Kranz
+1 On 06/22/2015 04:23 PM, Matthew Treinish wrote: Hi Everyone, I'd like to propose we add Jordan Pittier (jordanP) to the tempest core team. Jordan has been a steady contributor and reviewer on tempest over the past few cycles and he's been actively engaged in the Tempest community. Jordan

[openstack-dev] [qa] Status of account creds in the [identity] section of tempest.conf

2015-06-18 Thread David Kranz
We had a discussion about this at the qa meeting today around the following proposal: tl;dr The test accounts feature provides the same functionality as the embedded credentials. We should deprecate the account information embedded directly in tempest.conf in favor of test-accounts, and

[openstack-dev] [QA] Meeting Thursday June 18th at 17:00 UTC

2015-06-17 Thread David Kranz
to add an item to the agenda. To help people figure out what time 17:00 UTC is in other timezones tomorrow's meeting will be at: 13:00 EDT 02:00 JST 02:30 ACST 19:00 CEST 12:00 CDT 10:00 PDT -David Kranz __ OpenStack

Re: [openstack-dev] The Nova API in Kilo and Beyond

2015-06-05 Thread David Kranz
On 06/05/2015 07:32 AM, Sean Dague wrote: One of the things we realized at the summit was that we'd been working through a better future for the Nova API for the past 5 cycles, gotten somewhere quite useful, but had really done a poor job on communicating what was going on and why, and where

[openstack-dev] [QA] Meeting Thursday June 4th at 17:00 UTC

2015-06-03 Thread David Kranz
to add an item to the agenda. To help people figure out what time 17:00 UTC is in other timezones tomorrow's meeting will be at: 13:00 EDT 02:00 JST 02:30 ACST 19:00 CEST 12:00 CDT 10:00 PDT -David Kranz __ OpenStack Development

[openstack-dev] [qa] SafeConfigParser.write duplicates defaults: bug or feature?

2015-06-02 Thread David Kranz
The verify_tempest_config script has an option to write a new conf file. I noticed that when you do this, the items in DEFAULT are duplicated in every section that is written. Looking at the source I can see why this happens. I guess it is not harmful but is this considered a bug in the write

Re: [openstack-dev] [qa] Need volunteers for tempest bug triage

2015-06-01 Thread David Kranz
On 05/30/2015 09:15 AM, Kashyap Chamarthy wrote: On Sat, May 30, 2015 at 03:52:02PM +0300, Yaroslav Lobankov wrote: Hi everyone, Is it possible for other people (not only core reviewers) to participate in bug triage? I would like to help in doing this. Absolutely. There's no such silly rule

[openstack-dev] [qa] Need volunteers for tempest bug triage

2015-05-29 Thread David Kranz
The rotation has gotten a bit thin, and the untriaged bug count growing, with no one signing up for this past week: https://etherpad.openstack.org/p/qa-bug-triage-rotation It would help if every core reviewer could be doing this every other month. Getting some more sign ups would be very

Re: [openstack-dev] [new][cloudpulse] Announcing a project to HealthCheck OpenStack deployments

2015-05-13 Thread David Kranz
On 05/13/2015 09:06 AM, Simon Pasquier wrote: Hello, Like many others commented before, I don't quite understand how unique are the Cloudpulse use cases. For operators, I got the feeling that existing solutions fit well: - Traditional monitoring tools (Nagios, Zabbix, ) are necessary

Re: [openstack-dev] [new][cloudpulse] Announcing a project to HealthCheck OpenStack deployments

2015-05-13 Thread David Kranz
On 05/13/2015 09:51 AM, Simon Pasquier wrote: On Wed, May 13, 2015 at 3:27 PM, David Kranz dkr...@redhat.com mailto:dkr...@redhat.com wrote: On 05/13/2015 09:06 AM, Simon Pasquier wrote: Hello, Like many others commented before, I don't quite understand how unique

Re: [openstack-dev] [api] Changing 403 Forbidden to 400 Bad Request for OverQuota was: [nova] Which error code should we return when OverQuota

2015-05-06 Thread David Kranz
On 05/06/2015 02:07 PM, Jay Pipes wrote: Adding [api] topic. API WG members, please do comment. On 05/06/2015 08:01 AM, Sean Dague wrote: On 05/06/2015 07:11 AM, Chris Dent wrote: On Wed, 6 May 2015, Sean Dague wrote: All other client errors, just be a 400. And use the emerging error

Re: [openstack-dev] [swift] Go! Swift!

2015-04-30 Thread David Kranz
On 04/30/2015 12:52 PM, Jay Pipes wrote: On 04/30/2015 12:40 PM, John Dickinson wrote: Swift is a scalable and durable storage engine for storing unstructured data. It's been proven time and time again in production in clusters all over the world. We in the Swift developer community are

Re: [openstack-dev] [qa] trimming down Tempest smoke tag

2015-04-28 Thread David Kranz
On 04/28/2015 06:38 AM, Sean Dague wrote: The Tempest Smoke tag was originally introduced to provide a quick view of your OpenStack environment to ensure that a few basic things were working. It was intended to be fast. However, during Icehouse the smoke tag was repurposed as a way to let

Re: [openstack-dev] [all] Question for the TC candidates

2015-04-24 Thread David Kranz
On 04/24/2015 10:42 AM, Chris Dent wrote: On Fri, 24 Apr 2015, Ed Leafe wrote: I read the downstream to mean what you refer to as people who deploy workloads on them. In this context, I saw the operators as the end-users of the work the devs do. If that gave the impression that I don't care

[openstack-dev] [qa] official clients and tempest

2015-04-08 Thread David Kranz
Since tempest no longer uses the official clients as a literal code dependency, except for the cli tests which are being removed, the clients have been dropping from requirements.txt. But when debugging issues uncovered by tempest, or when debugging tempest itself, it is useful to use the cli

Re: [openstack-dev] [qa] official clients and tempest

2015-04-08 Thread David Kranz
On 04/08/2015 02:36 PM, Matthew Treinish wrote: On Wed, Apr 08, 2015 at 01:08:03PM -0400, David Kranz wrote: Since tempest no longer uses the official clients as a literal code dependency, except for the cli tests which are being removed, the clients have been dropping from requirements.txt

[openstack-dev] [qa] [devstack] Confusion about member roles in tempest/devstack

2015-04-06 Thread David Kranz
There have been a number of changes in tempest recently that seem to coordinate with devstack that are a bit unclear. The following values are defined in tempest config as defaults: [auth] # Roles to assign to all users created by tempest (list value) #tempest_roles = [object-storage] # Role

Re: [openstack-dev] [qa] [devstack] Confusion about member roles in tempest/devstack

2015-04-06 Thread David Kranz
On 04/06/2015 03:14 PM, Matthew Treinish wrote: On Mon, Apr 06, 2015 at 02:25:14PM -0400, David Kranz wrote: There have been a number of changes in tempest recently that seem to coordinate with devstack that are a bit unclear. Well, the issue was that before tempest was making all sorts

Re: [openstack-dev] [QA] [rally] How will Tempest discover/run tests migrated to specific projects?

2015-03-31 Thread David Kranz
On 03/30/2015 10:44 AM, Matthew Treinish wrote: On Mon, Mar 30, 2015 at 12:21:18PM +0530, Rohan Kanade wrote: Since tests can now be removed from Tempest https://wiki.openstack.org/wiki/QA/Tempest-test-removal and migrated to their specific projects. Does Tempest plan to discover/run these

[openstack-dev] [qa] Question about is_admin_available()

2015-03-10 Thread David Kranz
In the process of writing a unit test for this I discovered that it can call out to keystone for a token with some configurations through the call to get_configured_credentials. This surprised me since I thought it would just check for the necessary admin credentials in either tempest.conf or

[openstack-dev] [heat][qa] Forward plan for heat scenario tests

2015-03-09 Thread David Kranz
Since test_server_cfn_init was recently moved from tempest to the heat functional tests, there are no subclasses of OrchestrationScenarioTest. If there is no plan to add any more heat scenario tests to tempest I would like to remove that class. So I want to confirm that future scenario tests

Re: [openstack-dev] [QA] testing implementation-specific features not covered by OpenStack APIs

2015-03-03 Thread David Kranz
On 03/03/2015 11:28 AM, Radoslaw Zarzynski wrote: As we know Tempest provides many great tests for verification of conformance with OpenStack interfaces - the tempest/api directory is full of such useful stuff. However, regarding the #1422728 ticket [1] (dependency on private HTTP header of

Re: [openstack-dev] [all] Re-evaluating the suitability of the 6 month release cycle

2015-02-24 Thread David Kranz
On 02/24/2015 09:37 AM, Chris Dent wrote: On Tue, 24 Feb 2015, Sean Dague wrote: That also provides a very concrete answer to will people show up. Because if they do, and we get this horizontal refactoring happening, then we get to the point of being able to change release cadences faster. If

Re: [openstack-dev] [Nova][Tempest] Tempest will deny extra properties on Nova v2/v2.1 API

2015-02-24 Thread David Kranz
On 02/24/2015 06:55 AM, Ken'ichi Ohmichi wrote: Hi Ghanshyam, 2015-02-24 20:28 GMT+09:00 GHANSHYAM MANN ghanshyamm...@gmail.com: On Tue, Feb 24, 2015 at 6:48 PM, Ken'ichi Ohmichi ken1ohmi...@gmail.com wrote: Hi Nova team is developing Nova v2.1 API + microversions in this cycle, and the

[openstack-dev] [qa][swift] Signature of return values in tempest swift client

2015-02-13 Thread David Kranz
Almost all of the OpenStack REST apis return little of user value in the response headers, with json bodies containing the returned data. The tempest client methods had been returning two values with one always being ignored. To clean that up before moving the service clients to tempest-lib,

Re: [openstack-dev] [stable] juno is fubar in the gate

2015-02-10 Thread David Kranz
On 02/10/2015 10:35 AM, Matthew Treinish wrote: On Tue, Feb 10, 2015 at 11:19:20AM +0100, Thierry Carrez wrote: Joe, Matt Matthew: I hear your frustration with broken stable branches. With my vulnerability management team member hat, responsible for landing patches there with a strict

Re: [openstack-dev] [stable] juno is fubar in the gate

2015-02-10 Thread David Kranz
On 02/10/2015 12:20 PM, Jeremy Stanley wrote: On 2015-02-10 11:50:28 -0500 (-0500), David Kranz wrote: [...] I would rather give up branchless tempest than the ability for real distributors/deployers/operators to collaborate on stable branches. [...] Keep in mind that branchless tempest came

Re: [openstack-dev] [OpenStack Foundation] Finding people to work on the EC2 API in Nova

2015-02-06 Thread David Kranz
On 02/06/2015 07:49 AM, Sean Dague wrote: On 02/06/2015 07:39 AM, Alexandre Levine wrote: Rushi, We're adding new tempest tests into our stackforge-api/ec2-api. The review will appear in a couple of days. These tests will be good for running against both nova/ec2-api and stackforge/ec2-api. As

Re: [openstack-dev] [QA] Prototype of the script for Tempest auto-configuration

2015-02-04 Thread David Kranz
rebased the patch on master and ran the script. The script was finished without any errors and the tempest.conf was generated! Of course, this patch needs a lot of work, but the idea looks very cool! Also I would like to thank David Kranz for his working on initial version

[openstack-dev] [QA] Meeting Thursday January 15th at 17:00 UTC

2015-01-14 Thread David Kranz
UTC is in other timezones tomorrow's meeting will be at: 12:00 EST 02:00 JST 03:30 ACDT 18:00 CET 11:00 CST 09:00 PST -David Kranz __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev

Re: [openstack-dev] [QA] Moving tempest clients to tempest-lib (was Meeting Thursday January 8th at 22:00 UTC)

2015-01-13 Thread David Kranz
tests either through inheritance or delegation. Is that different than your vision? -David --- 2015-01-08 2:44 GMT+09:00 David Kranz dkr...@redhat.com: Hi everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, January 8th at 22:00 UTC

[openstack-dev] [QA] Meeting Thursday January 8th at 22:00 UTC

2015-01-07 Thread David Kranz
UTC is in other timezones tomorrow's meeting will be at: 17:00 EST 07:00 JST 08:30 ACDT 23:00 CET 16:00 CST 14:00 PST -David Kranz ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo

[openstack-dev] [all] Proper use of 'git review -R'

2014-12-30 Thread David Kranz
Many times when I review a revision of an existing patch, I can't see just the change from the previous version due to other rebases. The git-review documentation mentions this issue and suggests using -R to make life easier for reviewers when submitting new revisions. Can some one explain

Re: [openstack-dev] [all] Proper use of 'git review -R'

2014-12-30 Thread David Kranz
On 12/30/2014 11:37 AM, Jeremy Stanley wrote: On 2014-12-30 09:46:35 -0500 (-0500), David Kranz wrote: [...] Can some one explain when we should *not* use -R after doing 'git commit --amend'? [...] In the standard workflow this should never be necessary. The default behavior in git-review

[openstack-dev] [qa] tempest stable/icehouse builds are broken

2014-12-29 Thread David Kranz
Some kind of regression has caused stable/icehouse builds to fail, and hence prevents any code from merging in tempest. This is being tracked at https://bugs.launchpad.net/python-heatclient/+bug/1405579. Jeremy (fungi) provided a hacky work-around here https://review.openstack.org/#/c/144347/

[openstack-dev] [qa] neutron client returns one value has finally merged

2014-12-19 Thread David Kranz
Neutron patches can resume as normal. Thanks for the patience. -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [qa] Please do not merge neutron test changes until client returns one value is merged

2014-12-17 Thread David Kranz
This https://review.openstack.org/#/c/141152/ gets rid of the useless second return value from neutron client methods according to this spec: https://github.com/openstack/qa-specs/blob/master/specs/clients-return-one-value.rst. Because the client and test changes have to be in the same patch,

[openstack-dev] Reason for mem/vcpu ratio in default flavors

2014-12-11 Thread David Kranz
Perhaps this is a historical question, but I was wondering how the default OpenStack flavor size ratio of 2/1 was determined? According to http://aws.amazon.com/ec2/instance-types/, ec2 defines the flavors for General Purpose (M3) at about 3.7/1, with Compute Intensive (C3) at about 1.9/1 and

[openstack-dev] [qa] branchless tempest and the use of 'all' for extensions in tempest.conf

2014-12-03 Thread David Kranz
A recent proposed test to tempest was making explicit calls to the nova extension discovery api rather than using test.requires_ext. The reason was because we configure tempest.conf in the gate as 'all' for extensions, and the test involved an extension that was new in Juno. So the icehouse

[openstack-dev] Plans for removing xml support

2014-11-12 Thread David Kranz
Code has started going into tempest for several projects (nova,neutron,keystone) to allow removal of xml support in kilo. There have been many (heated) off and on threads on this list over the years. I'm sure many projects would like to do this, but there is evidence that not all have an

Re: [openstack-dev] [QA][All] Prelude to functional testing summit discussions

2014-10-30 Thread David Kranz
On 10/30/2014 07:49 AM, Sean Dague wrote: On 10/29/2014 12:30 PM, Matthew Treinish wrote: Hi everyone, Before we start the larger discussion at summit next week about the future of testing in OpenStack - specifically about spinning up functional testing and how it relates to tempest - I would

Re: [openstack-dev] [QA][All] Prelude to functional testing summit discussions

2014-10-30 Thread David Kranz
On 10/30/2014 09:52 AM, Sean Dague wrote: On 10/30/2014 09:33 AM, David Kranz wrote: On 10/30/2014 07:49 AM, Sean Dague wrote: On 10/29/2014 12:30 PM, Matthew Treinish wrote: Hi everyone, Before we start the larger discussion at summit next week about the future of testing in OpenStack

Re: [openstack-dev] [QA][All] Prelude to functional testing summit discussions

2014-10-30 Thread David Kranz
On 10/30/2014 11:12 AM, Sean Dague wrote: On 10/30/2014 10:47 AM, Eoghan Glynn wrote: Matthew wrote: This would have the advantage of making tempest slimmer for every project and begin the process of getting projects to take responsibility for their functional testing rather than relying on

Re: [openstack-dev] [Ceilometer] [qa] [oslo] Declarative HTTP Tests

2014-10-24 Thread David Kranz
On 10/23/2014 06:27 AM, Chris Dent wrote: I've proposed a spec to Ceilometer https://review.openstack.org/#/c/129669/ for a suite of declarative HTTP tests that would be runnable both in gate check jobs and in local dev environments. There's been some discussion that this may be generally

Re: [openstack-dev] [all] periodic jobs for master

2014-10-22 Thread David Kranz
On 10/22/2014 06:07 AM, Thierry Carrez wrote: Ihar Hrachyshka wrote: [...] For stable branches, we have so called periodic jobs that are triggered once in a while against the current code in a stable branch, and report to openstack-stable-maint@ mailing list. An example of failing periodic job

Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model

2014-09-24 Thread David Kranz
On 09/24/2014 02:48 PM, Clint Byrum wrote: Excerpts from Robert Collins's message of 2014-09-23 21:14:47 -0700: No one helped me edit this :) http://rbtcollins.wordpress.com/2014/09/24/what-poles-for-the-tent/ I hope I haven't zoned out and just channelled someone else here ;) This sounds

Re: [openstack-dev] [qa] Tempest Bug triage

2014-09-12 Thread David Kranz
On 09/12/2014 05:11 AM, Kashyap Chamarthy wrote: On Thu, Sep 11, 2014 at 03:52:56PM -0400, David Kranz wrote: So we had a Bug Day this week and the results were a bit disappointing due to lack of participation. We went from 124 New bugs to 75. There were also many cases where bugs referred

Re: [openstack-dev] Kilo Cycle Goals Exercise

2014-09-11 Thread David Kranz
On 09/11/2014 07:32 AM, Eoghan Glynn wrote: As you all know, there has recently been several very active discussions around how to improve assorted aspects of our development process. One idea that was brought up is to come up with a list of cycle goals/project priorities for Kilo [0]. To

[openstack-dev] [qa] Tempest Bug triage

2014-09-11 Thread David Kranz
So we had a Bug Day this week and the results were a bit disappointing due to lack of participation. We went from 124 New bugs to 75. There were also many cases where bugs referred to logs that no longer existed. This suggests that we really need to keep up with bug triage in real time. Since

[openstack-dev] [qa] Reminder: Tempest Bug Day: Tuesday September 9 (tomorrow)

2014-09-08 Thread David Kranz
It's been a while since we had a bug day. We now have 121 (now 124) NEW bugs: https://bugs.launchpad.net/tempest/+bugs?field.searchtext=field.status%3Alist=NEWorderby=-importance The first order of business is to triage these bugs. This is a large enough number that I hesitate to mention

Re: [openstack-dev] [qa][all][Heat] Packaging of functional tests

2014-09-05 Thread David Kranz
On 09/05/2014 12:10 PM, Matthew Treinish wrote: On Fri, Sep 05, 2014 at 09:42:17AM +1200, Steve Baker wrote: On 05/09/14 04:51, Matthew Treinish wrote: On Thu, Sep 04, 2014 at 04:32:53PM +0100, Steven Hardy wrote: On Thu, Sep 04, 2014 at 10:45:59AM -0400, Jay Pipes wrote: On 08/29/2014 05:15

[openstack-dev] [qa] Tempest Bug Day: Tuesday September 9

2014-09-02 Thread David Kranz
It's been a while since we had a bug day. We now have 121 NEW bugs: https://bugs.launchpad.net/tempest/+bugs?field.searchtext=field.status%3Alist=NEWorderby=-importance The first order of business is to triage these bugs. This is a large enough number that I hesitate to mention anything else,

[openstack-dev] [qa] Lack of consistency in returning response from tempest clients

2014-08-29 Thread David Kranz
While reviewing patches for moving response checking to the clients, I noticed that there are places where client methods do not return any value. This is usually, but not always, a delete method. IMO, every rest client method should return at least the response. Some services return just the

Re: [openstack-dev] [qa] Lack of consistency in returning response from tempest clients

2014-08-29 Thread David Kranz
On 08/29/2014 10:56 AM, Sean Dague wrote: On 08/29/2014 10:19 AM, David Kranz wrote: While reviewing patches for moving response checking to the clients, I noticed that there are places where client methods do not return any value. This is usually, but not always, a delete method. IMO, every

Re: [openstack-dev] [all] gate debugging

2014-08-27 Thread David Kranz
On 08/27/2014 02:54 PM, Sean Dague wrote: Note: thread intentionally broken, this is really a different topic. On 08/27/2014 02:30 PM, Doug Hellmann wrote: On Aug 27, 2014, at 1:30 PM, Chris Dent chd...@redhat.com wrote: On Wed, 27 Aug 2014, Doug Hellmann wrote: I have found it immensely

Re: [openstack-dev] [all] gate debugging

2014-08-27 Thread David Kranz
On 08/27/2014 03:43 PM, Sean Dague wrote: On 08/27/2014 03:33 PM, David Kranz wrote: On 08/27/2014 02:54 PM, Sean Dague wrote: Note: thread intentionally broken, this is really a different topic. On 08/27/2014 02:30 PM, Doug Hellmann wrote: On Aug 27, 2014, at 1:30 PM, Chris Dent chd

Re: [openstack-dev] [qa][all][Heat] Packaging of functional tests

2014-08-26 Thread David Kranz
On 08/26/2014 10:14 AM, Zane Bitter wrote: Steve Baker has started the process of moving Heat tests out of the Tempest repository and into the Heat repository, and we're looking for some guidance on how they should be packaged in a consistent way. Apparently there are a few projects already

Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-26 Thread David Kranz
On 08/26/2014 10:04 AM, Doug Hellmann wrote: On Aug 26, 2014, at 5:13 AM, Thierry Carrez thie...@openstack.org wrote: OK, now that we have evacuated the terminology issue (we'll use liaison or janitor or secretary, not czar), and side-stepped the offtopic development (this is not about

Re: [openstack-dev] [all] The future of the integrated release

2014-08-21 Thread David Kranz
On 08/21/2014 02:39 PM, gordon chung wrote: The point I've been making is that by the TC continuing to bless only the Ceilometer project as the OpenStack Way of Metering, I think we do a disservice to our users by picking a winner in a space that is clearly still unsettled. can we avoid

Re: [openstack-dev] [all] The future of the integrated release

2014-08-21 Thread David Kranz
On 08/21/2014 04:12 PM, Clint Byrum wrote: Excerpts from David Kranz's message of 2014-08-21 12:45:05 -0700: On 08/21/2014 02:39 PM, gordon chung wrote: The point I've been making is that by the TC continuing to bless only the Ceilometer project as the OpenStack Way of Metering, I think we do

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library

2014-08-20 Thread David Kranz
On 08/18/2014 04:57 PM, Matthew Treinish wrote: On Sat, Aug 16, 2014 at 06:27:19PM +0200, Marc Koderer wrote: Hi all, Am 15.08.2014 um 23:31 schrieb Jay Pipes jaypi...@gmail.com: I suggest that tempest should be the name of the import'able library, and that the integration tests themselves

[openstack-dev] Implications of moving functional tests to projects (was Re: Which program for Rally)

2014-08-12 Thread David Kranz
Changing subject line to continue thread about new $subj On 08/12/2014 08:56 AM, Doug Hellmann wrote: On Aug 11, 2014, at 12:00 PM, David Kranz dkr...@redhat.com mailto:dkr...@redhat.com wrote: On 08/06/2014 05:48 PM, John Griffith wrote: I have to agree with Duncan here. I also don't

Re: [openstack-dev] Which program for Rally

2014-08-11 Thread David Kranz
On 08/06/2014 05:48 PM, John Griffith wrote: I have to agree with Duncan here. I also don't know if I fully understand the limit in options. Stress test seems like it could/should be different (again overlap isn't a horrible thing) and I don't see it as siphoning off resources so not sure of

Re: [openstack-dev] Which program for Rally

2014-08-11 Thread David Kranz
On 08/11/2014 04:21 PM, Matthew Treinish wrote: I apologize for the delay in my response to this thread, between travelling and having a stuck 'a' key on my laptop this is the earliest I could respond. I opted for a separate branch on this thread to summarize my views and I'll respond

[openstack-dev] [QA] Meeting Thursday July 31 at 17:00UTC

2014-07-29 Thread David Kranz
to the agenda. To help people figure out what time 17:00 UTC is in other timezones, Thursday's meeting will be at: 13:00 EDT 02:00 JST 02:30 ACST 19:00 CEST 12:00 CDT 10:00 PDT -David Kranz ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org

[openstack-dev] [qa] The role of an abstract client in tempest

2014-07-25 Thread David Kranz
Even as a core contributor for several years, it has never been clear what the scope of these tests should be. As we move forward with the necessity of moving functional testing to projects, we need to answer this question for real, understanding that part of the mission for these tests now is

Re: [openstack-dev] Thoughts on the patch test failure rate and moving forward

2014-07-25 Thread David Kranz
On 07/25/2014 10:01 AM, Steven Hardy wrote: On Wed, Jul 23, 2014 at 02:39:47PM -0700, James E. Blair wrote: snip * Put the burden for a bunch of these tests back on the projects as functional tests. Basically a custom devstack environment that a project can create with a set of

[openstack-dev] [cinder][qa] cinder client versions and tempest

2014-07-24 Thread David Kranz
I noticed that the cinder list-extensions url suffix is underneath the v1/v2 in the GET url but the returned result is the same either way. Some of the returned items have v1 in the namespace, and others v2. Also, in tempest, there is a single config section for cinder and only a single

Re: [openstack-dev] [tc][rally][qa] Application for a new OpenStack Program: Performance and Scalability

2014-07-22 Thread David Kranz
On 07/22/2014 10:44 AM, Sean Dague wrote: Honestly, I'm really not sure I see this as a different program, but is really something that should be folded into the QA program. I feel like a top level effort like this is going to lead to a lot of duplication in the data analysis that's currently

Re: [openstack-dev] [gate] The gate: a failure analysis

2014-07-21 Thread David Kranz
On 07/21/2014 04:13 PM, Jay Pipes wrote: On 07/21/2014 02:03 PM, Clint Byrum wrote: Thanks Matthew for the analysis. I think you missed something though. Right now the frustration is that unrelated intermittent bugs stop your presumably good change from getting in. Without gating, the result

Re: [openstack-dev] [QA] Proposed Changes to Tempest Core

2014-07-21 Thread David Kranz
+1 On Jul 21, 2014, at 6:37 PM, Matthew Treinish mtrein...@kortar.org wrote: Hi Everyone, I would like to propose 2 changes to the Tempest core team: First, I'd like to nominate Andrea Fritolli to the Tempest core team. Over the past cycle Andrea has been steadily become more actively

Re: [openstack-dev] [qa][all] Branchless Tempest beyond pure-API tests, impact on backporting policy

2014-07-10 Thread David Kranz
On 07/10/2014 08:53 AM, Matthew Treinish wrote: On Thu, Jul 10, 2014 at 08:37:40AM -0400, Eoghan Glynn wrote: Note that the notifications that capture these resource state transitions are a long-standing mechanism in openstack that ceilometer has depended on from the very outset. I don't think

Re: [openstack-dev] [qa][all] Branchless Tempest beyond pure-API tests, impact on backporting policy

2014-07-10 Thread David Kranz
On 07/10/2014 09:47 AM, Thierry Carrez wrote: Hi! There is a lot of useful information in that post (even excluding the part brainstorming solutions) and it would be a shame if it was lost in a sub-thread. Do you plan to make a blog post, or reference wiki page, out of this ? Back to the

Re: [openstack-dev] [qa] [rfc] move scenario tests to tempest client

2014-07-10 Thread David Kranz
On 07/10/2014 08:08 AM, Frittoli, Andrea (HP Cloud) wrote: ++ The ugly monkey patch approach is still working fine for my downstream testing, but that's something I'd be happy to get rid of. Something that may be worth considering is to have an abstraction layer on top of tempest clients, to

[openstack-dev] [qa] Strange error in non-isolated periodic tests

2014-07-10 Thread David Kranz
I was trying to bring the periodic non-isolated jobs back to health. One problem with them is all the scenario tests fail with Captured traceback: 2014-06-26 07:00:42.312

[openstack-dev] [keystone][qa] Returning 203 in keystone v2 list apis?

2014-07-03 Thread David Kranz
While moving success response code checking in tempest to the client, I noticed that exactly one of the calls to list users for a tenant checked for 200 or 203. Looking at http://docs.openstack.org/api/openstack-identity-service/2.0/content/, it seems that most of the list apis can return 203.

[openstack-dev] [qa][nova] Please run 'check experimental' on changes to nova v3 tests in tempest

2014-07-02 Thread David Kranz
Due to the status of nova v3, to save time, running the tempest v3 tests has been moved out of the gate/check jobs to the experimental queue. So please run 'check experimental' on v3-related patches. -David ___ OpenStack-dev mailing list

[openstack-dev] [qa] checking success response in clients

2014-06-30 Thread David Kranz
We approved https://github.com/openstack/qa-specs/blob/master/specs/client-checks-success.rst which recommends that checking of correct success codes be moved to the tempest clients. This has been done for the image tests but not others yet. But new client/test code coming in should definitely

[openstack-dev] [marconi][qa] marconi check run in tempest is on experimental queue

2014-06-23 Thread David Kranz
Please remember to do 'check experimental' after uploading new marconi patches in tempest. -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Rethink how we manage projects? (was Gate proposal - drop Postgresql configurations in the gate)

2014-06-16 Thread David Kranz
On 06/16/2014 05:33 AM, Thierry Carrez wrote: David Kranz wrote: [...] There is a different way to do this. We could adopt the same methodology we have now around gating, but applied to each project on its own branch. These project branches would be integrated into master at some frequency

[openstack-dev] [qa] Clarification of policy for qa-specs around adding new tests

2014-06-16 Thread David Kranz
I have been reviewing some of these specs and sense a lack of clarity around what is expected. In the pre-qa-specs world we did not want tempest blueprints to be used by projects to track their tempest test submissions because the core review team did not want to have to spend a lot of time

Re: [openstack-dev] Rethink how we manage projects? (was Gate proposal - drop Postgresql configurations in the gate)

2014-06-13 Thread David Kranz
On 06/13/2014 07:31 AM, Sean Dague wrote: On 06/13/2014 02:36 AM, Mark McLoughlin wrote: On Thu, 2014-06-12 at 22:10 -0400, Dan Prince wrote: On Thu, 2014-06-12 at 08:06 -0400, Sean Dague wrote: We're definitely deep into capacity issues, so it's going to be time to start making tougher

[openstack-dev] [qa] Meaning of 204 from DELETE apis

2014-06-12 Thread David Kranz
Tempest has a number of tests in various services for deleting objects that mostly return 204. Many, but not all, of these tests go on to check that the resource was actually deleted but do so in different ways. Sometimes they go into a timeout loop waiting for a GET on the object to fail.

Re: [openstack-dev] [qa] Meaning of 204 from DELETE apis

2014-06-12 Thread David Kranz
On 06/12/2014 05:27 PM, Jay Pipes wrote: On 06/12/2014 05:17 PM, David Kranz wrote: Tempest has a number of tests in various services for deleting objects that mostly return 204. Many, but not all, of these tests go on to check that the resource was actually deleted but do so in different ways

[openstack-dev] [qa] Should Assignee, Milestone really be in the qa-spec?

2014-06-11 Thread David Kranz
While reviewing some specs I noticed that I had put myself down for more Juno-2 work than is likely to be completed. I suspect this will happen routinely with many folks. Also, assignees may change. This information is not really part of the spec at all. Since we are still using blueprints to

Re: [openstack-dev] [qa] Should Assignee, Milestone really be in the qa-spec?

2014-06-11 Thread David Kranz
On 06/11/2014 03:50 PM, Matthew Treinish wrote: On Wed, Jun 11, 2014 at 03:17:48PM -0400, David Kranz wrote: While reviewing some specs I noticed that I had put myself down for more Juno-2 work than is likely to be completed. I suspect this will happen routinely with many folks. Also, assignees

Re: [openstack-dev] [qa] shared review dashboard proposal

2014-06-10 Thread David Kranz
On 06/09/2014 02:24 PM, Sean Dague wrote: On 06/09/2014 01:38 PM, David Kranz wrote: On 06/02/2014 06:57 AM, Sean Dague wrote: Towards the end of the summit there was a discussion about us using a shared review dashboard to see if a common view by the team would help accelerate people looking

Re: [openstack-dev] [qa] shared review dashboard proposal

2014-06-09 Thread David Kranz
On 06/02/2014 06:57 AM, Sean Dague wrote: Towards the end of the summit there was a discussion about us using a shared review dashboard to see if a common view by the team would help accelerate people looking at certain things. I spent some time this weekend working on a tool to make building

Re: [openstack-dev] [qa][nova] Status of v3 tests in tempest

2014-05-20 Thread David Kranz
On 05/20/2014 03:19 PM, Christopher Yeoh wrote: On Tue, May 20, 2014 at 8:58 PM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: On 05/19/2014 11:49 PM, Christopher Yeoh wrote: - if/else inlined in tests based on the microversion mode that is being tested at the

[openstack-dev] [qa][nova] Status of v3 tests in tempest

2014-05-19 Thread David Kranz
It seems the nova team decided in Atlanta that v3 as currently understood is never going to exist: https://etherpad.openstack.org/p/juno-nova-v3-api. There are a number of patches in flight that tweak how we handle supporting both v2/v3 in tempest to reduce duplication. We need to decide what

Re: [openstack-dev] [qa][nova] Status of v3 tests in tempest

2014-05-19 Thread David Kranz
in tempests own rest client as well. andrea -Original Message- From: David Kranz [mailto:dkr...@redhat.com] Sent: 19 May 2014 10:49 To: OpenStack Development Mailing List Subject: [openstack-dev] [qa][nova] Status of v3 tests in tempest It seems the nova team decided in Atlanta that v3

Re: [openstack-dev] [qa][nova] Status of v3 tests in tempest

2014-05-19 Thread David Kranz
, David Kranz wrote: On 05/19/2014 01:24 PM, Frittoli, Andrea (HP Cloud) wrote: Thanks for bringing this up. We won't be testing v3 in Juno, but we'll need coverage for v2.1. In my understanding will be a v2 compatible API - so including proxy to glance cinder and neutron - but with micro-versions

Re: [openstack-dev] [qa] Checking for return codes in tempest client calls

2014-05-09 Thread David Kranz
On 05/09/2014 11:29 AM, Matthew Treinish wrote: On Thu, May 08, 2014 at 09:50:03AM -0400, David Kranz wrote: On 05/07/2014 10:48 AM, Ken'ichi Ohmichi wrote: Hi Sean, 2014-05-07 23:28 GMT+09:00 Sean Dague s...@dague.net: On 05/07/2014 10:23 AM, Ken'ichi Ohmichi wrote: Hi David, 2014-05-07

Re: [openstack-dev] [qa] Checking for return codes in tempest client calls

2014-05-08 Thread David Kranz
On 05/07/2014 10:48 AM, Ken'ichi Ohmichi wrote: Hi Sean, 2014-05-07 23:28 GMT+09:00 Sean Dague s...@dague.net: On 05/07/2014 10:23 AM, Ken'ichi Ohmichi wrote: Hi David, 2014-05-07 22:53 GMT+09:00 David Kranz dkr...@redhat.com: I just looked at a patch https://review.openstack.org/#/c/90310

[openstack-dev] [qa] Checking for return codes in tempest client calls

2014-05-07 Thread David Kranz
I just looked at a patch https://review.openstack.org/#/c/90310/3 which was given a -1 due to not checking that every call to list_hosts returns 200. I realized that we don't have a shared understanding or policy about this. We need to make sure that each api is tested to return the right

Re: [openstack-dev] [qa] Checking for return codes in tempest client calls

2014-05-07 Thread David Kranz
On 05/07/2014 10:07 AM, Duncan Thomas wrote: On 7 May 2014 14:53, David Kranz dkr...@redhat.com wrote: I just looked at a patch https://review.openstack.org/#/c/90310/3 which was given a -1 due to not checking that every call to list_hosts returns 200. I realized that we don't have a shared

Re: [openstack-dev] [Openstack-dev][qa] Running tempest tests against existing openstack installation

2014-05-05 Thread David Kranz
On 05/05/2014 02:26 AM, Swapnil Kulkarni wrote: Hello, I am trying to run tempest tests against an existing openstack deployment. I have configured tempest.conf for the environment details. But when I execute run_tempest.sh, it does not run any tests. Although when I run testr run, the

Re: [openstack-dev] [all] Branchless Tempest QA Spec - final draft

2014-05-01 Thread David Kranz
On 05/01/2014 11:36 AM, Matthew Treinish wrote: On Thu, May 01, 2014 at 06:18:10PM +0900, Ken'ichi Ohmichi wrote: # Sorry for sending this again, previous mail was unreadable. 2014-04-28 11:54 GMT+09:00 Ken'ichi Ohmichi ken1ohmi...@gmail.com: This is also why there are a bunch of nova v2

  1   2   >