Re: [openstack-dev] Which program for Rally

2014-08-11 Thread David Kranz
On 08/06/2014 05:48 PM, John Griffith wrote: I have to agree with Duncan here. I also don't know if I fully understand the limit in options. Stress test seems like it could/should be different (again overlap isn't a horrible thing) and I don't see it as siphoning off resources so not sure of

Re: [openstack-dev] Which program for Rally

2014-08-11 Thread David Kranz
On 08/11/2014 04:21 PM, Matthew Treinish wrote: I apologize for the delay in my response to this thread, between travelling and having a stuck 'a' key on my laptop this is the earliest I could respond. I opted for a separate branch on this thread to summarize my views and I'll respond

[openstack-dev] Implications of moving functional tests to projects (was Re: Which program for Rally)

2014-08-12 Thread David Kranz
Changing subject line to continue thread about new $subj On 08/12/2014 08:56 AM, Doug Hellmann wrote: On Aug 11, 2014, at 12:00 PM, David Kranz dkr...@redhat.com mailto:dkr...@redhat.com wrote: On 08/06/2014 05:48 PM, John Griffith wrote: I have to agree with Duncan here. I also don't

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library

2014-08-20 Thread David Kranz
On 08/18/2014 04:57 PM, Matthew Treinish wrote: On Sat, Aug 16, 2014 at 06:27:19PM +0200, Marc Koderer wrote: Hi all, Am 15.08.2014 um 23:31 schrieb Jay Pipes jaypi...@gmail.com: I suggest that tempest should be the name of the import'able library, and that the integration tests themselves

Re: [openstack-dev] [all] The future of the integrated release

2014-08-21 Thread David Kranz
On 08/21/2014 02:39 PM, gordon chung wrote: The point I've been making is that by the TC continuing to bless only the Ceilometer project as the OpenStack Way of Metering, I think we do a disservice to our users by picking a winner in a space that is clearly still unsettled. can we avoid

Re: [openstack-dev] [all] The future of the integrated release

2014-08-21 Thread David Kranz
On 08/21/2014 04:12 PM, Clint Byrum wrote: Excerpts from David Kranz's message of 2014-08-21 12:45:05 -0700: On 08/21/2014 02:39 PM, gordon chung wrote: The point I've been making is that by the TC continuing to bless only the Ceilometer project as the OpenStack Way of Metering, I think we do

Re: [openstack-dev] [qa][all][Heat] Packaging of functional tests

2014-08-26 Thread David Kranz
On 08/26/2014 10:14 AM, Zane Bitter wrote: Steve Baker has started the process of moving Heat tests out of the Tempest repository and into the Heat repository, and we're looking for some guidance on how they should be packaged in a consistent way. Apparently there are a few projects already

Re: [openstack-dev] [all] [ptls] The Czar system, or how to scale PTLs

2014-08-26 Thread David Kranz
On 08/26/2014 10:04 AM, Doug Hellmann wrote: On Aug 26, 2014, at 5:13 AM, Thierry Carrez thie...@openstack.org wrote: OK, now that we have evacuated the terminology issue (we'll use liaison or janitor or secretary, not czar), and side-stepped the offtopic development (this is not about

Re: [openstack-dev] [all] gate debugging

2014-08-27 Thread David Kranz
On 08/27/2014 02:54 PM, Sean Dague wrote: Note: thread intentionally broken, this is really a different topic. On 08/27/2014 02:30 PM, Doug Hellmann wrote: On Aug 27, 2014, at 1:30 PM, Chris Dent chd...@redhat.com wrote: On Wed, 27 Aug 2014, Doug Hellmann wrote: I have found it immensely

Re: [openstack-dev] [all] gate debugging

2014-08-27 Thread David Kranz
On 08/27/2014 03:43 PM, Sean Dague wrote: On 08/27/2014 03:33 PM, David Kranz wrote: On 08/27/2014 02:54 PM, Sean Dague wrote: Note: thread intentionally broken, this is really a different topic. On 08/27/2014 02:30 PM, Doug Hellmann wrote: On Aug 27, 2014, at 1:30 PM, Chris Dent chd

[openstack-dev] [qa] Lack of consistency in returning response from tempest clients

2014-08-29 Thread David Kranz
While reviewing patches for moving response checking to the clients, I noticed that there are places where client methods do not return any value. This is usually, but not always, a delete method. IMO, every rest client method should return at least the response. Some services return just the

Re: [openstack-dev] [qa] Lack of consistency in returning response from tempest clients

2014-08-29 Thread David Kranz
On 08/29/2014 10:56 AM, Sean Dague wrote: On 08/29/2014 10:19 AM, David Kranz wrote: While reviewing patches for moving response checking to the clients, I noticed that there are places where client methods do not return any value. This is usually, but not always, a delete method. IMO, every

[openstack-dev] [qa] Tempest Bug Day: Tuesday September 9

2014-09-02 Thread David Kranz
It's been a while since we had a bug day. We now have 121 NEW bugs: https://bugs.launchpad.net/tempest/+bugs?field.searchtext=field.status%3Alist=NEWorderby=-importance The first order of business is to triage these bugs. This is a large enough number that I hesitate to mention anything else,

Re: [openstack-dev] [qa][all][Heat] Packaging of functional tests

2014-09-05 Thread David Kranz
On 09/05/2014 12:10 PM, Matthew Treinish wrote: On Fri, Sep 05, 2014 at 09:42:17AM +1200, Steve Baker wrote: On 05/09/14 04:51, Matthew Treinish wrote: On Thu, Sep 04, 2014 at 04:32:53PM +0100, Steven Hardy wrote: On Thu, Sep 04, 2014 at 10:45:59AM -0400, Jay Pipes wrote: On 08/29/2014 05:15

[openstack-dev] [qa] Reminder: Tempest Bug Day: Tuesday September 9 (tomorrow)

2014-09-08 Thread David Kranz
It's been a while since we had a bug day. We now have 121 (now 124) NEW bugs: https://bugs.launchpad.net/tempest/+bugs?field.searchtext=field.status%3Alist=NEWorderby=-importance The first order of business is to triage these bugs. This is a large enough number that I hesitate to mention

Re: [openstack-dev] Kilo Cycle Goals Exercise

2014-09-11 Thread David Kranz
On 09/11/2014 07:32 AM, Eoghan Glynn wrote: As you all know, there has recently been several very active discussions around how to improve assorted aspects of our development process. One idea that was brought up is to come up with a list of cycle goals/project priorities for Kilo [0]. To

[openstack-dev] [qa] Tempest Bug triage

2014-09-11 Thread David Kranz
So we had a Bug Day this week and the results were a bit disappointing due to lack of participation. We went from 124 New bugs to 75. There were also many cases where bugs referred to logs that no longer existed. This suggests that we really need to keep up with bug triage in real time. Since

Re: [openstack-dev] [qa] Tempest Bug triage

2014-09-12 Thread David Kranz
On 09/12/2014 05:11 AM, Kashyap Chamarthy wrote: On Thu, Sep 11, 2014 at 03:52:56PM -0400, David Kranz wrote: So we had a Bug Day this week and the results were a bit disappointing due to lack of participation. We went from 124 New bugs to 75. There were also many cases where bugs referred

Re: [openstack-dev] [qa] [Solum] [tempest] Use of pecan test framework in functional tests

2013-12-10 Thread David Kranz
On 12/10/2013 04:12 PM, Russell Bryant wrote: On 12/10/2013 04:10 PM, Georgy Okrokvertskhov wrote: Hi, In Solum project we are currently creating tests environments for future test. We split unit tests and functional tests in order to use tempest framework from the beginning. Tempest

Re: [openstack-dev] [ironic][qa] How will ironic tests run in tempest?

2013-12-11 Thread David Kranz
On 12/10/2013 08:41 PM, Devananda van der Veen wrote: Tue, Dec 10, 2013 at 12:43 PM, David Kranz dkr...@redhat.com mailto:dkr...@redhat.com wrote: On 12/09/2013 01:37 PM, Devananda van der Veen wrote: On Fri, Dec 6, 2013 at 2:13 PM, Clark Boylan clark.boy...@gmail.com

[openstack-dev] [cinder][qa] release notes for cinder v1 to v2?

2013-12-16 Thread David Kranz
Sorry for lost subject in last message. Is there a document that describes the api changes from v1 to v2, similar to the one documenting nova v2 to v3? -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org

[openstack-dev] [savanna][qa] How will tempest tests run?

2013-12-16 Thread David Kranz
So it's great to see a submission of savanna tests for tempest. We would like to see these tests run before reviewing them. Is the intent that savanna will be enabled by default in devstack? If not, then I guess there will need to be separate savanna jobs. I see that right now there are

Re: [openstack-dev] [qa][ceilometer] Who can be a asked to help review tempest tests?

2013-12-16 Thread David Kranz
this was because I didn't know it was possible to ask all of you like that! -David On Mon, Dec 16, 2013 at 5:33 PM, David Kranz dkr...@redhat.com mailto:dkr...@redhat.com wrote: Ceilometer team, we are reviewing tempest tests and hope to see more. The tempest review team is hoping

[openstack-dev] [qa][neutron] Please remember to mark changed tests as 'smoke'

2013-12-23 Thread David Kranz
We like all code submitted to tempest to actually run. Since the neutron gate jobs are still running only smoke tests, please mark any test that is added or whose code has changed as smoke. Note that 'smoke' has no real other meaning now since it was applied haphazardly in the first place and

Re: [openstack-dev] [Tempest][qa] Adding tags to commit messages

2013-12-24 Thread David Kranz
On 12/24/2013 06:32 AM, Sean Dague wrote: On 12/24/2013 01:47 AM, Yair Fried wrote: Hi, Suggestion: Please consider tagging your Tempest commit messages the same way you do your mails in the mailing list Explanation: Since tempest is a single project testing multiple Openstack project we

Re: [openstack-dev] [Ceilometer][Tempest]Tracking of blueprints and changes

2013-12-27 Thread David Kranz
On 12/27/2013 05:27 AM, Nadya Privalova wrote: Hello guys! I hope all of you are enjoying the holidays! But I'd like to raise a Tempest question. Again. I hope this email will not be lost after vacations :) After the summit we decided to track all tests that are being created for Ceilometer

Re: [openstack-dev] [nova] minimum review period for functional changes that break backwards compatibility

2013-12-29 Thread David Kranz
On 12/28/2013 11:14 AM, Tim Bell wrote: I think there is a need for an incompatible change review process which includes more of the community than just those performing the code reviews. This kind of change can cause a lot of disruption for those of us running clouds so it is great to see

Re: [openstack-dev] [Tempest][qa] Adding tags to commit messages

2013-12-30 Thread David Kranz
On 12/29/2013 07:45 PM, Jeremy Stanley wrote: On 2013-12-29 15:09:24 -0500 (-0500), David Kranz wrote: [...] Looking at the docs I see the warning that you can't put this in the search field so I tried putting it directly in the url like the other parameters but it was ignored. Is there indeed

[openstack-dev] [all project] log files and logstash

2013-12-30 Thread David Kranz
In case any one other than me didn't know this, the log files that are indexed and searchable in logstash are not the same as the set of files that you see in the logs directory in jenkins, but only those that have an entry in

Re: [openstack-dev] [Ceilometer][Tempest]Tracking of blueprints and changes

2013-12-30 Thread David Kranz
On 12/27/2013 05:27 AM, Nadya Privalova wrote: Hello guys! I hope all of you are enjoying the holidays! But I'd like to raise a Tempest question. Again. I hope this email will not be lost after vacations :) After the summit we decided to track all tests that are being created for Ceilometer

[openstack-dev] [qa] Negative test generation

2014-01-02 Thread David Kranz
I took a little time to think some more about this and pushed a little working prototype of some ideas I had https://review.openstack.org/#/c/64733/1. Comments about the approach are most welcome. Note that I minimized any refactoring of the existing hierarchy for this prorotype. -David

Re: [openstack-dev] [nova] minimum review period for functional changes that break backwards compatibility

2014-01-03 Thread David Kranz
On 01/03/2014 08:52 AM, Thierry Carrez wrote: Tim Bell wrote: Is there a mechanism to tag changes as being potentially more appropriate for the more ops related profiles ? I'm thinking more when someone proposes a change they suspect could have an operations impact, they could highlight this

Re: [openstack-dev] [qa] Have multiple config files

2014-01-06 Thread David Kranz
On 01/06/2014 05:14 AM, Shweta Jain wrote: Hello there , I am working on Tempest and is new to it . while I am using tempest.config but would like to know is there a way I can use multiple config files in tempest . My intention to use the multiple configs is purely for running the test in

[openstack-dev] [qa] negative test generation and testscenarios

2014-01-06 Thread David Kranz
Thanks to all who looked at https://review.openstack.org/#/c/64733/. There were a few minor issues I will address but the biggest one was the suggestion to run each variation as a separate test case using testscenarios. After looking into that I see a problem with this use case. Many of these

Re: [openstack-dev] [nova][neutron][qa] top gate bugs: a plea for help

2014-01-12 Thread David Kranz
On 01/11/2014 05:06 PM, Russell Bryant wrote: On 01/11/2014 11:38 AM, Sean Dague wrote: 3) (still testing) https://review.openstack.org/#/c/65805/ Right now when tempest runs in the devstack-gate jobs, it runs with concurrency=4 (run 4 tests at once). Unfortunately, it appears that this maxes

Re: [openstack-dev] [nova] [qa] api schema validation pattern changes

2014-01-12 Thread David Kranz
On 01/12/2014 10:14 PM, Matthew Treinish wrote: snip Last, is a question, is it possible to currently run the full API an build a json schema for it all? Or to query these validating schemas? We *really* want that over in tempest, so we can completely drop the manual creation of negative

[openstack-dev] [qa] [ceilometer] Can this be fixed

2014-01-13 Thread David Kranz
A bug was introduced before the holidays that made tempest stop failing successful builds that have new log ERRORs. We are ready to close that bug but unfortunately some new ones slipped through as indicated in https://bugs.launchpad.net/ceilometer/+bug/1268730 I just filed. It would be great

Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely

2014-01-14 Thread David Kranz
On 01/14/2014 10:37 AM, Thomas Goirand wrote: On 01/13/2014 10:38 PM, Sean Dague wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to

Re: [openstack-dev] [devstack] libvirt default log level

2014-01-15 Thread David Kranz
On 01/15/2014 11:56 AM, Steven Dake wrote: Hi, Ken'ichi Omichi submitted a change [1] in devstack to change the default log level to 1 for libvirt. This results in continual spam to /var/log/messages in my development system, even after exiting devstack. The spam looks like: Jan 14

Re: [openstack-dev] [QA] The future of nosetests with Tempest

2014-01-17 Thread David Kranz
On 01/16/2014 10:56 PM, Matthew Treinish wrote: Hi everyone, With some recent changes made to Tempest compatibility with nosetests is going away. We've started using newer features that nose just doesn't support. One example of this is that we've started using testscenarios and we're planning

Re: [openstack-dev] [qa] RE: [Tempest - Stress Test] : implement a full SSH connection on ssh_floating.py and improve it

2014-01-17 Thread David Kranz
On 01/17/2014 09:06 AM, Koderer, Marc wrote: Hi Julien, most of the cases in tempest/stress are already covered by exiting tests in /api or /scenario. The only thing that is missing is the decorator on them. BTW here is the Etherpad from the summit talk that we had:

Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-22 Thread David Kranz
On 01/22/2014 03:19 PM, Jay Pipes wrote: On Tue, 2014-01-21 at 01:15 -0500, Yair Fried wrote: I seem to be unable to convey my point using generalization, so I will give a specific example: I would like to have update dns server as an additional network scenario. Currently I could add it to

Re: [openstack-dev] [nova] massive number of new errors in logs with oslo.messaging

2014-02-03 Thread David Kranz
On 02/03/2014 07:36 AM, Monty Taylor wrote: On 02/02/2014 08:33 PM, Jay Pipes wrote: On Sun, 2014-02-02 at 07:13 -0500, Sean Dague wrote: Just noticed this at the end of a successful run:

Re: [openstack-dev] [qa][tempest][rally] Rally Tempest integration: tempest.conf autogeneration

2014-02-12 Thread David Kranz
to work with tempest. We added first patch that adds base functionality to Rally [3]: https://review.openstack.org/#/c/70131/ At QA meeting I discussed it with David Kranz, as a result we agree that part of this functionality (tempest.conf generator cloud prepare), should be implemented inside

[openstack-dev] bad default values in conf files

2014-02-13 Thread David Kranz
I was recently bitten by a case where some defaults in keystone.conf were not appropriate for real deployment, and our puppet modules were not providing better values https://bugzilla.redhat.com/show_bug.cgi?id=1064061. Since there are hundreds (thousands?) of options across all the services.

[openstack-dev] [qa] Cleaning OpenStack resources

2014-02-19 Thread David Kranz
I was looking at https://review.openstack.org/#/c/73274/1 which makes it configurable whether a brute-force cleanup of resources is done after success. This got my wondering how this should really be done. As admin, there are some resources that can be cleaned and some that I don't know how.

[openstack-dev] [qa] Re: [openstack-qa] Not Able to Run Tempest API Tests

2014-02-20 Thread David Kranz
On 02/20/2014 05:58 AM, om prakash pandey wrote: I am not able to run Tempest API tests. The typical ERROR I am getting is Connection Timed Out. When checking into the logs I found out that tempest is trying to access the admin URL which is a private IP for our deployment. Now, Tempest is

[openstack-dev] [qa] Does scenario.test_minimum_basic need to upload ami images?

2014-02-20 Thread David Kranz
Running this test in tempest requires an ami image triple to be on the disk where tempest is running in order for the test to upload it. It would be a lot easier if this test could use a simple image file instead. That image file could even be obtained from the cloud being tested while

Re: [openstack-dev] [qa] Does scenario.test_minimum_basic need to upload ami images?

2014-02-20 Thread David Kranz
On 02/20/2014 04:53 PM, Sean Dague wrote: On 02/20/2014 04:31 PM, David Kranz wrote: Running this test in tempest requires an ami image triple to be on the disk where tempest is running in order for the test to upload it. It would be a lot easier if this test could use a simple image file

[openstack-dev] [qa] Can't interpret failure from XenServer CI

2014-02-23 Thread David Kranz
XenServer reported failure on https://review.openstack.org/#/c/73704. The pointer to the logs should look like a jenkins failure but does not in two ways. First, there is no console log, with a directory of other log files next to it. Second, the .gz log files are not set up to be downloaded

Re: [openstack-dev] [qa] Can't interpret failure from XenServer CI

2014-02-23 Thread David Kranz
mention 'citrix recheck'. I only discovered that through a web search. In general, the more we can do to establish uniformity of reviewer experience across all third-party ci, the more likely it is that reviewers will examine failures. -David From: David

Re: [openstack-dev] [qa][nova] Ownership and path to schema definitions

2014-03-06 Thread David Kranz
On 03/05/2014 10:25 PM, Christopher Yeoh wrote: On Tue, 04 Mar 2014 13:31:07 -0500 David Kranz dkr...@redhat.com wrote: I think it would be a good time to have at least an initial discussion about the requirements for theses schemas and where they will live. The next step in tempest around

[openstack-dev] [qa] Python 3.3 patches (using six)

2014-03-10 Thread David Kranz
There are a number of patches up for review that make various changes to use six apis instead of Python 2 constructs. While I understand the desire to get a head start on getting Tempest to run in Python 3, I'm not sure it makes sense to do this work piecemeal until we are near ready to

[openstack-dev] [qa] Tests depending on conf boolean values that are False in the gate

2014-03-12 Thread David Kranz
I have recently reviewed a few patches (around testing consoles) that add new tests whose execution depends on False config options or introduce new ones that are False by default. The result is that the new tests do not run in any job. Since we have a policy that there should only be code in

Re: [openstack-dev] [qa] Python 3.3 patches (using six)

2014-03-13 Thread David Kranz
On 03/13/2014 10:50 AM, Joe Hakim Rahme wrote: On 10 Mar 2014, at 22:54, David Kranz dkr...@redhat.com wrote: There are a number of patches up for review that make various changes to use six apis instead of Python 2 constructs. While I understand the desire to get a head start on getting

Re: [openstack-dev] [qa] Python 3.3 patches (using six)

2014-03-13 Thread David Kranz
On 03/13/2014 04:56 PM, Sean Dague wrote: On 03/13/2014 04:29 PM, David Kranz wrote: On 03/13/2014 10:50 AM, Joe Hakim Rahme wrote: On 10 Mar 2014, at 22:54, David Kranz dkr...@redhat.com wrote: There are a number of patches up for review that make various changes to use six apis instead

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread David Kranz
On 03/20/2014 06:15 AM, Sean Dague wrote: On 03/20/2014 05:49 AM, Nadya Privalova wrote: Hi all, First of all, thanks for your suggestions! To summarize the discussions here: 1. We are not going to install Mongo (because is's wrong ?) We are not going to install Mongo not from base

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

2014-03-20 Thread David Kranz
On 03/20/2014 12:31 PM, Sean Dague wrote: On 03/20/2014 11:35 AM, David Kranz wrote: On 03/20/2014 06:15 AM, Sean Dague wrote: On 03/20/2014 05:49 AM, Nadya Privalova wrote: Hi all, First of all, thanks for your suggestions! To summarize the discussions here: 1. We are not going to install

Re: [openstack-dev] [nova] Backwards incompatible API changes

2014-03-20 Thread David Kranz
On 03/20/2014 11:05 AM, Solly Ross wrote: I concur. I suspect people/organizations who are doing CD *probably* won't mind such a change as much as the people who use the versioned releases will mind backwards-incompatibility. Correct me if I'm wrong, but doing CD requires a certain willingness

Re: [openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest

2014-03-21 Thread David Kranz
On 03/20/2014 04:19 PM, Rochelle.RochelleGrober wrote: -Original Message- From: Malini Kamalambal [mailto:malini.kamalam...@rackspace.com] Sent: Thursday, March 20, 2014 12:13 PM 'project specific functional testing' in the Marconi context is treating Marconi as a complete system,

Re: [openstack-dev] [nova] Backwards incompatible API changes

2014-03-21 Thread David Kranz
On 03/21/2014 05:04 AM, Christopher Yeoh wrote: On Thu, 20 Mar 2014 15:45:11 -0700 Dan Smith d...@danplanet.com wrote: I know that our primary delivery mechanism is releases right now, and so if we decide to revert before this gets into a release, that's cool. However, I think we need to be

Re: [openstack-dev] [nova] Backwards incompatible API changes

2014-03-21 Thread David Kranz
. There was a related discussion about getting more eyes on changes that propose to be excepted from the API stability guidelines here http://lists.openstack.org/pipermail/openstack-dev/2014-January/023254.html -David On Mar 21, 2014, at 10:34 AM, David Kranz dkr...@redhat.com mailto:dkr

[openstack-dev] [qa] [ironic] Need tempest reviews from ironic team

2014-03-31 Thread David Kranz
I was reviewing some ironic changes that are more than a week old and do not have any reviews from the ironic team. Having at least one review from the ironic team would be very helpful. -David ___ OpenStack-dev mailing list

Re: [openstack-dev] [qa] [ironic] Need tempest reviews from ironic team

2014-03-31 Thread David Kranz
very valuable in neutron and I'm sure it would be here as well. I will certainly ping in #openstack-ironic if there are any specific issues. -David On Mon, Mar 31, 2014 at 8:47 AM, David Kranz dkr...@redhat.com mailto:dkr...@redhat.com wrote: I was reviewing some ironic changes

Re: [openstack-dev] [RFC] Tempest without branches

2014-04-04 Thread David Kranz
On 04/04/2014 07:37 AM, Sean Dague wrote: An interesting conversation has cropped up over the last few days in -qa and -infra which I want to bring to the wider OpenStack community. When discussing the use of Tempest as part of the Defcore validation we came to an interesting question: Why does

Re: [openstack-dev] [Hyper-V] Havana status

2013-10-11 Thread David Kranz
On 10/11/2013 02:03 PM, Alessandro Pilotti wrote: On Oct 11, 2013, at 19:29 , Russell Bryant rbry...@redhat.com mailto:rbry...@redhat.com wrote: On 10/11/2013 12:04 PM, John Griffith wrote: [... snip ...] Talking about new community involvements, newcomers are getting very

Re: [openstack-dev] Thanks for fixing my patch

2013-10-11 Thread David Kranz
On 10/11/2013 02:34 PM, Clint Byrum wrote: Recently in the TripleO meeting we identified situations where we need to make it very clear that it is ok to pick up somebody else's patch and finish it. We are broadly distributed, time-zone-wise, and I know other teams working on OpenStack projects

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread David Kranz
Thanks, Steve. I suggested a new directory because we really need to have more complete tests of the client libs since they are not tied to particular OpenStack releases and we claim the current libs should work with older releases. That said, I did not realize the intent was to do more than

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-18 Thread David Kranz
On 10/18/2013 12:17 PM, Boris Pavlovic wrote: John, Actually seems like a pretty good suggestion IMO, at least something worth some investigation and consideration before quickly discounting it. Rather than that's not what tempest is, maybe it's something tempest could do. Don't know, not

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread David Kranz
On 10/18/2013 01:54 PM, Sean Dague wrote: On 10/18/2013 11:59 AM, Dolph Mathews wrote: On Fri, Oct 18, 2013 at 10:34 AM, Steven Hardy sha...@redhat.com mailto:sha...@redhat.com wrote: Hi all, Starting a thread to discuss $subject, as requested in:

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread David Kranz
On 10/20/2013 03:03 PM, Robert Collins wrote: On 21 October 2013 07:36, Alex Gaynor alex.gay...@gmail.com wrote: There's several issues involved in doing automated regression checking for benchmarks: - You need a platform which is stable. Right now all our CI runs on virtualized instances, and

Re: [openstack-dev] [ceilometer] [qa] Ceilometer ERRORS in normal runs

2013-10-22 Thread David Kranz
On 10/22/2013 10:19 AM, Sean Dague wrote: On 10/21/2013 10:27 AM, Neal, Phil wrote: Sean, we currently have a BP out there to investigate basic tempest integration and I think this might fall under the same umbrella. Unfortunately I've not been able to free up my development time for it, but

Re: [openstack-dev] [ceilometer] [qa] Ceilometer ERRORS in normal runs

2013-10-23 Thread David Kranz
On 10/23/2013 05:08 PM, Rochelle.Grober wrote: John Griffith wrote: On Wed, Oct 23, 2013 at 8:47 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: On 10/23/2013 10:40 AM, John Griffith wrote: On Sun, Oct 20, 2013 at 7:38 AM, Sean Dague s...@dague.net

[openstack-dev] [qa] Issue with tests of host admin api

2013-10-25 Thread David Kranz
A patch was submitted with some new tests of this api https://review.openstack.org/#/c/49778/. I gave a -1 because if a negative test to shutdown a host fails, a compute node will be shutdown. The author thinks this test should be part of tempest. My issue was that we should not have tempest

Re: [openstack-dev] [qa] Issue with tests of host admin api

2013-10-25 Thread David Kranz
On 10/25/2013 09:10 AM, Sean Dague wrote: On 10/25/2013 08:39 AM, David Kranz wrote: A patch was submitted with some new tests of this api https://review.openstack.org/#/c/49778/. I gave a -1 because if a negative test to shutdown a host fails, a compute node will be shutdown. The author thinks

Re: [openstack-dev] Re : welcoming new committers (was Re: When is it okay for submitters to say 'I don't want to add tests' ?)

2013-11-01 Thread David Kranz
On 10/31/2013 10:36 PM, Jeremy Stanley wrote: On 2013-10-31 22:45:56 + (+), Romain Hardouin wrote: Adding a message for new comers is a good idea. I am a new Horizon contributor, some of my fixes have been merged (thanks to Upstream University :-) and reviewers of course) but I still

[openstack-dev] [qa] Major change to tempest.conf.sample coming

2013-11-13 Thread David Kranz
This is a heads up that soon we will be auto-generating the tempest.conf.sample from the tempest code that uses oslo.config. Following in the footsteps of nova, this should reduce bugs around failures to keep the config code and the sample conf file in sync manually. So when you add a new item

Re: [openstack-dev] [openstack-qa] [qa][nova] The document for the changes from Nova v2 api to v3

2013-11-13 Thread David Kranz
On 11/13/2013 08:30 AM, Alex Xu wrote: Hi, guys This is the document for the changes from Nova v2 api to v3: https://wiki.openstack.org/wiki/NovaAPIv2tov3 I will appreciate if anyone can help for review it. Another problem comes up - how to keep the doc updated. So can we ask people, who

[openstack-dev] [qa] Tracking development of scenario tests

2013-11-13 Thread David Kranz
It was clear at the summit that there is a pressing need for more scenario tests. A number of folks have volunteered to participate so we need a way to track progress and avoid duplication. We have not had great satisfaction using either bugs or blueprints, so Sean and I are proposing a more

Re: [openstack-dev] [qa] Tracking development of scenario tests

2013-11-18 Thread David Kranz
@lists.openstack.org Subject: Re: [openstack-dev] [qa] Tracking development of scenario tests On 11/14/2013 12:24 AM, David Kranz wrote: 1. Developer checks in the zeroth version of a scenario test as work in progress. It contains a description of the test, and possibly work items. This will claim

[openstack-dev] [qa] Plan for failing successful tempest jobs when new ERRORs appear in logs

2013-11-18 Thread David Kranz
So we are close to being able to start doing this. The current whitelist is here https://github.com/openstack/tempest/blob/master/etc/whitelist.yaml. I have a find-errors script that watches for successful builds and pulls out the non-whitelisted errors. For the past few weeks I have been

Re: [openstack-dev] [Neutron][Tempest] Tempest API test for Neutron LBaaS

2013-11-20 Thread David Kranz
On 11/19/2013 09:40 PM, Sean Dague wrote: On 11/17/2013 09:55 PM, Eugene Nikanorov wrote: Hi folks, I'm working on major change to Neutron LBaaS API, obviously it will break existing tempest API tests for LBaaS. What would be the right process to deal with this? I guess I can't just push fixed

[openstack-dev] [all project] Time to fail tempest gate jobs when new log errors appear

2013-11-27 Thread David Kranz
tl;dr Soon, perhaps next week, tempest gate jobs will start failing if there are any ERROR lines in the logs that are not matched by an entry in https://github.com/openstack/tempest/blob/master/etc/whitelist.yaml. There is an exception for neutron because more work needs to be done there for

Re: [openstack-dev] excessively difficult to support both iso8601 0.1.4 and 0.1.8 as deps

2013-11-27 Thread David Kranz
On 11/27/2013 12:46 PM, Alan Pevec wrote: 2013/11/27 Sean Dague s...@dague.net: The problem is you can't really support both iso8601 was dormant for years, and the revived version isn't compatible with the old version. So supporting both means basically forking iso8601 and maintaining you own

[openstack-dev] [ceilometer][qa] Punting ceilometer from whitelist

2013-11-29 Thread David Kranz
In preparing to fail builds with log errors I have been trying to make things easier for projects by maintaining a whitelist. But these bugs in ceilometer are coming in so fast that I can't keep up. So I am just putting .* in the white list for any cases I find before gate failing is turned

[openstack-dev] [qa] Pinging people for reviews in IRC

2013-11-29 Thread David Kranz
Folks, I understand that the review latency can be too long. We just added two core reviewers and I am sure we can do better still. But please, if you feel you must ping some one by name for a review, do so in #openstack-qa rather than pinging on a private channel. That way other people might

[openstack-dev] [all-projects] PLEASE READ: Code to fail builds on new log errors is merging

2013-12-02 Thread David Kranz
As mentioned several times on this list, https://review.openstack.org/#/c/58848/ is in the process of merging. Once it does, builds will start failing if there are log ERRORs that are not filtered out by https://github.com/openstack/tempest/blob/master/etc/whitelist.yaml. There are too many

[openstack-dev] [ceilometer][qa] log errors

2013-12-02 Thread David Kranz
So the last two bugs I filed were coming from check builds that were not merged. It was necessary to look at all builds to get to the point where gate failing on errors could be turned on. Here is the current whitelist. If you folks think the .*' entries are no longer needed then please let me

Re: [openstack-dev] [ceilometer][qa] Punting ceilometer from whitelist

2013-12-02 Thread David Kranz
On 12/02/2013 10:24 AM, Julien Danjou wrote: On Fri, Nov 29 2013, David Kranz wrote: In preparing to fail builds with log errors I have been trying to make things easier for projects by maintaining a whitelist. But these bugs in ceilometer are coming in so fast that I can't keep up. So I am

Re: [openstack-dev] [ceilometer][qa] Punting ceilometer from whitelist

2013-12-03 Thread David Kranz
On 12/03/2013 09:30 AM, Eoghan Glynn wrote: - Original Message - On 12/02/2013 10:24 AM, Julien Danjou wrote: On Fri, Nov 29 2013, David Kranz wrote: In preparing to fail builds with log errors I have been trying to make things easier for projects by maintaining a whitelist

Re: [openstack-dev] [qa] Moving the QA meeting time

2013-12-05 Thread David Kranz
On 12/05/2013 07:16 AM, Sean Dague wrote: On 12/05/2013 02:37 AM, Koderer, Marc wrote: Hi all! -Ursprüngliche Nachricht- Von: Kenichi Oomichi [mailto:oomi...@mxs.nes.nec.co.jp] Gesendet: Donnerstag, 5. Dezember 2013 01:37 An: OpenStack Development Mailing List (not for usage

Re: [openstack-dev] [qa][nova] The document for the changes from Nova v2 api to v3

2013-12-06 Thread David Kranz
On 11/13/2013 06:09 PM, Christopher Yeoh wrote: On Thu, Nov 14, 2013 at 7:52 AM, David Kranz dkr...@redhat.com mailto:dkr...@redhat.com wrote: On 11/13/2013 08:30 AM, Alex Xu wrote: Hi, guys This is the document for the changes from Nova v2 api to v3: https

[openstack-dev] [qa] Changes slipping through the nova v2-v3 port

2013-12-06 Thread David Kranz
I have been trying to review all of the nova v3 changes. Many of these patches have been around for awhile and have not kept up with changes that were made to the v2 tests after a v2 test file was copied to v3. I think any one submitting a patch to the nova v2 test code needs to file a bug

Re: [openstack-dev] [qa][nova] The document for the changes from Nova v2 api to v3

2013-12-06 Thread David Kranz
have info for the json dict part, not any changes to the url suffix or return values. Second, there really needs to be a release note to help users upgrade their apps from v2 to v3. -David --- 2013/12/7 David Kranz dkr...@redhat.com: On 11/13/2013 06:09 PM, Christopher Yeoh wrote: On Thu

[openstack-dev] [ironic][qa] How will ironic tests run in tempest?

2013-12-06 Thread David Kranz
It's great that tempest tests for ironic have been submitted! I was reviewing https://review.openstack.org/#/c/48109/ and noticed that the tests do not actually run. They are skipped because baremetal is not enabled. This is not terribly surprising but we have had a policy in tempest to only

[openstack-dev] [qa] Should Assignee, Milestone really be in the qa-spec?

2014-06-11 Thread David Kranz
While reviewing some specs I noticed that I had put myself down for more Juno-2 work than is likely to be completed. I suspect this will happen routinely with many folks. Also, assignees may change. This information is not really part of the spec at all. Since we are still using blueprints to

Re: [openstack-dev] [qa] Should Assignee, Milestone really be in the qa-spec?

2014-06-11 Thread David Kranz
On 06/11/2014 03:50 PM, Matthew Treinish wrote: On Wed, Jun 11, 2014 at 03:17:48PM -0400, David Kranz wrote: While reviewing some specs I noticed that I had put myself down for more Juno-2 work than is likely to be completed. I suspect this will happen routinely with many folks. Also, assignees

[openstack-dev] [qa] Meaning of 204 from DELETE apis

2014-06-12 Thread David Kranz
Tempest has a number of tests in various services for deleting objects that mostly return 204. Many, but not all, of these tests go on to check that the resource was actually deleted but do so in different ways. Sometimes they go into a timeout loop waiting for a GET on the object to fail.

Re: [openstack-dev] [qa] Meaning of 204 from DELETE apis

2014-06-12 Thread David Kranz
On 06/12/2014 05:27 PM, Jay Pipes wrote: On 06/12/2014 05:17 PM, David Kranz wrote: Tempest has a number of tests in various services for deleting objects that mostly return 204. Many, but not all, of these tests go on to check that the resource was actually deleted but do so in different ways

Re: [openstack-dev] Rethink how we manage projects? (was Gate proposal - drop Postgresql configurations in the gate)

2014-06-13 Thread David Kranz
On 06/13/2014 07:31 AM, Sean Dague wrote: On 06/13/2014 02:36 AM, Mark McLoughlin wrote: On Thu, 2014-06-12 at 22:10 -0400, Dan Prince wrote: On Thu, 2014-06-12 at 08:06 -0400, Sean Dague wrote: We're definitely deep into capacity issues, so it's going to be time to start making tougher

  1   2   >