Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-21 Thread Steven Hardy
On Mon, Oct 21, 2013 at 08:14:03AM -0400, Sean Dague wrote: > On 10/21/2013 05:19 AM, Steven Hardy wrote: > > >> > >>Definitely agree we should have plenty of end-to-end tests in the > >>gate, it's the reason we've got the scenario tests, to do exactly > >>this kind of through testing. > > > >Ok,

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-21 Thread Sean Dague
On 10/21/2013 05:19 AM, Steven Hardy wrote: Definitely agree we should have plenty of end-to-end tests in the gate, it's the reason we've got the scenario tests, to do exactly this kind of through testing. Ok, it seems like a potential solution which may keep all involved happy would be: - Ad

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-21 Thread Steven Hardy
On Sat, Oct 19, 2013 at 07:45:33AM -0400, Sean Dague wrote: > On 10/19/2013 04:46 AM, Steven Hardy wrote: > >On Fri, Oct 18, 2013 at 01:48:08PM -0400, Sean Dague wrote: > >>So v3 keystone API is one thing, but I'm a little concerned with > >>moving the client testing to Tempest haphazardly. If we

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-19 Thread Sean Dague
On 10/19/2013 04:46 AM, Steven Hardy wrote: On Fri, Oct 18, 2013 at 01:48:08PM -0400, Sean Dague wrote: So v3 keystone API is one thing, but I'm a little concerned with moving the client testing to Tempest haphazardly. If we are testing the API surface on the servers, the clients should be able

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-19 Thread Sean Dague
On 10/18/2013 09:10 PM, Adam Young wrote: On 10/18/2013 07:21 PM, Sean Dague wrote: On 10/18/2013 05:09 PM, Dolph Mathews wrote: On Fri, Oct 18, 2013 at 3:19 PM, David Stanek mailto:dsta...@dstanek.com>> wrote: On Fri, Oct 18, 2013 at 1:48 PM, Sean Dague mailto:s...@dague.net>> wrote:

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-19 Thread Steven Hardy
On Fri, Oct 18, 2013 at 01:48:08PM -0400, Sean Dague wrote: > So v3 keystone API is one thing, but I'm a little concerned with > moving the client testing to Tempest haphazardly. If we are testing > the API surface on the servers, the clients should be able to > correctly test all of this via a mo

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Adam Young
On 10/18/2013 07:21 PM, Sean Dague wrote: On 10/18/2013 05:09 PM, Dolph Mathews wrote: On Fri, Oct 18, 2013 at 3:19 PM, David Stanek mailto:dsta...@dstanek.com>> wrote: On Fri, Oct 18, 2013 at 1:48 PM, Sean Dague mailto:s...@dague.net>> wrote: On 10/18/2013 12:04 PM, Brant Knudso

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Sean Dague
On 10/18/2013 05:09 PM, Dolph Mathews wrote: On Fri, Oct 18, 2013 at 3:19 PM, David Stanek mailto:dsta...@dstanek.com>> wrote: On Fri, Oct 18, 2013 at 1:48 PM, Sean Dague mailto:s...@dague.net>> wrote: On 10/18/2013 12:04 PM, Brant Knudson wrote: 2) "git clone"ing th

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Dolph Mathews
On Fri, Oct 18, 2013 at 3:19 PM, David Stanek wrote: > > On Fri, Oct 18, 2013 at 1:48 PM, Sean Dague wrote: > >> On 10/18/2013 12:04 PM, Brant Knudson wrote: >> >>> >>> 2) "git clone"ing the keystoneclient doesn't work well with parallel >>> testing (we have a similar problem in our tests with o

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread David Stanek
On Fri, Oct 18, 2013 at 1:48 PM, Sean Dague wrote: > On 10/18/2013 12:04 PM, Brant Knudson wrote: > >> >> 2) "git clone"ing the keystoneclient doesn't work well with parallel >> testing (we have a similar problem in our tests with our "pristine" >> database backup) >> > > Can you go into the spec

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Adam Young
On 10/18/2013 01:54 PM, Sean Dague wrote: On 10/18/2013 11:59 AM, Dolph Mathews wrote: On Fri, Oct 18, 2013 at 10:34 AM, Steven Hardy mailto:sha...@redhat.com>> wrote: Hi all, Starting a thread to discuss $subject, as requested in: https://review.openstack.org/#/c/51558/ Fir

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread David Kranz
On 10/18/2013 01:54 PM, Sean Dague wrote: On 10/18/2013 11:59 AM, Dolph Mathews wrote: On Fri, Oct 18, 2013 at 10:34 AM, Steven Hardy mailto:sha...@redhat.com>> wrote: Hi all, Starting a thread to discuss $subject, as requested in: https://review.openstack.org/#/c/51558/ Fir

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Sean Dague
On 10/18/2013 11:59 AM, Dolph Mathews wrote: On Fri, Oct 18, 2013 at 10:34 AM, Steven Hardy mailto:sha...@redhat.com>> wrote: Hi all, Starting a thread to discuss $subject, as requested in: https://review.openstack.org/#/c/51558/ First a bit of background. I wrote a keystone

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Adam Young
On 10/18/2013 12:04 PM, Brant Knudson wrote: To provide a bit more background... Keystone has a bunch of keystoneclient tests for the v2 API. These tests actually "git clone" a version of keystoneclient (master, essex-3, and 0.1.1)[0], to use for testing. Maybe at some point the tests were jus

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Sean Dague
On 10/18/2013 12:04 PM, Brant Knudson wrote: To provide a bit more background... Keystone has a bunch of keystoneclient tests for the v2 API. These tests actually "git clone" a version of keystoneclient (master, essex-3, and 0.1.1)[0], to use for testing. Maybe at some point the tests were just

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread David Kranz
Thanks, Steve. I suggested a new directory because we really need to have more complete tests of the client libs since they are not tied to particular OpenStack releases and we claim the current libs should work with older releases. That said, I did not realize the intent was to do more than te

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Brant Knudson
To provide a bit more background... Keystone has a bunch of keystoneclient tests for the v2 API. These tests actually "git clone" a version of keystoneclient (master, essex-3, and 0.1.1)[0], to use for testing. Maybe at some point the tests were just for client-server compatibility, but now they'r

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Dolph Mathews
On Fri, Oct 18, 2013 at 10:34 AM, Steven Hardy wrote: > Hi all, > > Starting a thread to discuss $subject, as requested in: > > https://review.openstack.org/#/c/51558/ > > First a bit of background. I wrote a keystoneclient patch, and ayoung > stated he'd like it tested via tempest before he'd a

[openstack-dev] [qa][keystone] Adding client library related tests to tempest

2013-10-18 Thread Steven Hardy
Hi all, Starting a thread to discuss $subject, as requested in: https://review.openstack.org/#/c/51558/ First a bit of background. I wrote a keystoneclient patch, and ayoung stated he'd like it tested via tempest before he'd ack it: https://review.openstack.org/#/c/48462/ So I spoke to ayoung