Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-21 Thread Steven Hardy
On Sat, Oct 19, 2013 at 07:45:33AM -0400, Sean Dague wrote: On 10/19/2013 04:46 AM, Steven Hardy wrote: On Fri, Oct 18, 2013 at 01:48:08PM -0400, Sean Dague wrote: So v3 keystone API is one thing, but I'm a little concerned with moving the client testing to Tempest haphazardly. If we are

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-21 Thread Sean Dague
On 10/21/2013 05:19 AM, Steven Hardy wrote: snip Definitely agree we should have plenty of end-to-end tests in the gate, it's the reason we've got the scenario tests, to do exactly this kind of through testing. Ok, it seems like a potential solution which may keep all involved happy would be:

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-21 Thread Steven Hardy
On Mon, Oct 21, 2013 at 08:14:03AM -0400, Sean Dague wrote: On 10/21/2013 05:19 AM, Steven Hardy wrote: snip Definitely agree we should have plenty of end-to-end tests in the gate, it's the reason we've got the scenario tests, to do exactly this kind of through testing. Ok, it seems like

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-19 Thread Steven Hardy
On Fri, Oct 18, 2013 at 01:48:08PM -0400, Sean Dague wrote: So v3 keystone API is one thing, but I'm a little concerned with moving the client testing to Tempest haphazardly. If we are testing the API surface on the servers, the clients should be able to correctly test all of this via a mock

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-19 Thread Sean Dague
On 10/19/2013 04:46 AM, Steven Hardy wrote: On Fri, Oct 18, 2013 at 01:48:08PM -0400, Sean Dague wrote: So v3 keystone API is one thing, but I'm a little concerned with moving the client testing to Tempest haphazardly. If we are testing the API surface on the servers, the clients should be

[openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Steven Hardy
Hi all, Starting a thread to discuss $subject, as requested in: https://review.openstack.org/#/c/51558/ First a bit of background. I wrote a keystoneclient patch, and ayoung stated he'd like it tested via tempest before he'd ack it: https://review.openstack.org/#/c/48462/ So I spoke to

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Dolph Mathews
On Fri, Oct 18, 2013 at 10:34 AM, Steven Hardy sha...@redhat.com wrote: Hi all, Starting a thread to discuss $subject, as requested in: https://review.openstack.org/#/c/51558/ First a bit of background. I wrote a keystoneclient patch, and ayoung stated he'd like it tested via tempest

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Brant Knudson
To provide a bit more background... Keystone has a bunch of keystoneclient tests for the v2 API. These tests actually git clone a version of keystoneclient (master, essex-3, and 0.1.1)[0], to use for testing. Maybe at some point the tests were just for client-server compatibility, but now they're

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread David Kranz
Thanks, Steve. I suggested a new directory because we really need to have more complete tests of the client libs since they are not tied to particular OpenStack releases and we claim the current libs should work with older releases. That said, I did not realize the intent was to do more than

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Sean Dague
On 10/18/2013 12:04 PM, Brant Knudson wrote: To provide a bit more background... Keystone has a bunch of keystoneclient tests for the v2 API. These tests actually git clone a version of keystoneclient (master, essex-3, and 0.1.1)[0], to use for testing. Maybe at some point the tests were just

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Adam Young
On 10/18/2013 12:04 PM, Brant Knudson wrote: To provide a bit more background... Keystone has a bunch of keystoneclient tests for the v2 API. These tests actually git clone a version of keystoneclient (master, essex-3, and 0.1.1)[0], to use for testing. Maybe at some point the tests were just

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Sean Dague
On 10/18/2013 11:59 AM, Dolph Mathews wrote: On Fri, Oct 18, 2013 at 10:34 AM, Steven Hardy sha...@redhat.com mailto:sha...@redhat.com wrote: Hi all, Starting a thread to discuss $subject, as requested in: https://review.openstack.org/#/c/51558/ First a bit of background. I

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread David Kranz
On 10/18/2013 01:54 PM, Sean Dague wrote: On 10/18/2013 11:59 AM, Dolph Mathews wrote: On Fri, Oct 18, 2013 at 10:34 AM, Steven Hardy sha...@redhat.com mailto:sha...@redhat.com wrote: Hi all, Starting a thread to discuss $subject, as requested in:

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Adam Young
On 10/18/2013 01:54 PM, Sean Dague wrote: On 10/18/2013 11:59 AM, Dolph Mathews wrote: On Fri, Oct 18, 2013 at 10:34 AM, Steven Hardy sha...@redhat.com mailto:sha...@redhat.com wrote: Hi all, Starting a thread to discuss $subject, as requested in:

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread David Stanek
On Fri, Oct 18, 2013 at 1:48 PM, Sean Dague s...@dague.net wrote: On 10/18/2013 12:04 PM, Brant Knudson wrote: 2) git cloneing the keystoneclient doesn't work well with parallel testing (we have a similar problem in our tests with our pristine database backup) Can you go into the

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Dolph Mathews
On Fri, Oct 18, 2013 at 3:19 PM, David Stanek dsta...@dstanek.com wrote: On Fri, Oct 18, 2013 at 1:48 PM, Sean Dague s...@dague.net wrote: On 10/18/2013 12:04 PM, Brant Knudson wrote: 2) git cloneing the keystoneclient doesn't work well with parallel testing (we have a similar problem in

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Sean Dague
On 10/18/2013 05:09 PM, Dolph Mathews wrote: On Fri, Oct 18, 2013 at 3:19 PM, David Stanek dsta...@dstanek.com mailto:dsta...@dstanek.com wrote: On Fri, Oct 18, 2013 at 1:48 PM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: On 10/18/2013 12:04 PM, Brant Knudson

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Adam Young
On 10/18/2013 07:21 PM, Sean Dague wrote: On 10/18/2013 05:09 PM, Dolph Mathews wrote: On Fri, Oct 18, 2013 at 3:19 PM, David Stanek dsta...@dstanek.com mailto:dsta...@dstanek.com wrote: On Fri, Oct 18, 2013 at 1:48 PM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: