Re: [openstack-dev] [openstack-ansible] Adding a scenario for shade's functional tests

[Jean-Philippe Evrard]

Hey,

My two cents.

JP

On 04/04/2017, 13:39, "Monty Taylor"  wrote:

Hey all!

I woke up early this morning and found myself thinking "I clearly don't
have enough work on my plate, why don't I add some more?"

I'd love to make a gate job that:
- Deploys a cloud with openstack-ansible
- Runs shade's functional tests against that cloud

How do you run these functional tests? Do you expect this as a simple tox run, 
a tempest scenario, or anything else?

Which makes me think it just needs to be an additional scenario perhaps?

I think a scenario is good, but we could consider this test as part of a basic 
end-to-end testing. If the tests aren’t too long, we could include them at the 
end of an AIO run to make sure everything “works as expected” for both tempest 
and shade. Sadly, our jobs are already long and hitting timeouts on some 
distros.

As I started to poke, I need to figure out the best way to accomplish
something. Namely, shade's functional tests expect a clouds.yaml that
defines two entires - one admin and one non-admin.

OSA already creates a clouds.yaml with admin creds, so that's done. And
there is already tempest setup that creates a demo user, so that's done
too. Here's where I could use some advice:

- What's the best way to plumb through an option to also write an entry
to clouds.yaml for the demo user? I'm thinking a boolean in
openstack-ansible-openstack_openrc like
"openrc_clouds_yml_add_demo_user" that can be wrapped around a second
entry in clouds.yaml.j2?

I’d agree with Jesse definition there: have vars defined in utility_all group 
vars (or like jesse said, something in the AIO prep).
These vars would:
- override the openstack_openrc default vars (we need to adapt the tasks in 
openstack_openrc “Create openrc” to include a loop + “Create clouds.yaml” to 
generate a more complete clouds.yaml according to the users defined in the dict)
- override the tempest user creation (we need to adapt the tasks in tempest to 
be a var + at the same time we could maybe move to ansible os_ modules there, 
that would also be a win for shade)

- Can we create the demo user and not run tempest? It's in the
os_tempest role at the moment. Should I split out the user creation bits
into a "test_setup" role that tempest depends on - and then also make a
shade_test role that also depends on the test_setup role?

I like this approach.
If you want a shorter path, you can still define in the user variables 
``tempest_run: False`` for the scenario, and we could have an include 
shade_run. I guess it all depends on shade_run content and if that deserves the 
burden of a role to maintain.
If you think shade_test deserves a role, we could transfer the maintenance 
burden to you by including it in the ansible-role-requirements, but not under 
the openstack-ansible umbrella, like we do for some external roles. We could 
then adapt the plays, or use an include_role inside tempest role for laziness.

- If we do a test_setup role, should that maybe just write a whole
additional clouds.yaml out that has the admin and the demo user so that
we don't have to put conditionals in places that also get used for prod
deploys?

We used to need the openrc role to happen before the tempest role IIRC.
We couldn’t have one role fits all. However, I see in that model that tempest 
or shade roles could both have a meta dep on openrc, and openrc being the 
source of truth for generating openrc like files (openrc + clouds.yaml). Openrc 
could then be used for production deploys to have a more complete clouds.yaml 
file.
Maybe it’s worth splitting the clouds.yaml with a secure.yaml in that case too. 
But that’s another topic.

I think that's about all the questions I have for now ...

Thanks!
Monty

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




Rackspace Limited is a company registered in England & Wales (company 
registered number 03897010) whose registered office is at 5 Millington Road, 
Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be 
viewed at www.rackspace.co.uk/legal/privacy-policy - This e-mail message may 
contain confidential or privileged information intended for the recipient. Any 
dissemination, distribution or copying of the enclosed material is prohibited. 
If you receive this transmission in error, please notify us immediately by 
e-mail at ab...@rackspace.com and delete the original message. Your cooperation 
is appreciated.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 

Re: [openstack-dev] [openstack-ansible] Adding a scenario for shade's functional tests

[Jesse Pretorius]

On 4/4/17, 1:39 PM, "Monty Taylor"  wrote:

>I'd love to make a gate job that:
>   - Deploys a cloud with openstack-ansible
>   - Runs shade's functional tests against that cloud

Fantastic. Let’s see how we can help.

>   Which makes me think it just needs to be an additional scenario perhaps?

Maybe. Our scenario’s, right now, are focused more around what 
Infrastructure/OpenStack services are deployed. If you want/need a larger set 
than our default ‘aio’ scenario, then yes a different scenario would need to be 
setup. The alternative, of course, is to override the confd_overrides var in 
your test setup with an ‘aio’ scenario of your own – but I think it’d be nice 
to rather bake whatever you need in so that we can help maintain it.
https://github.com/openstack/openstack-ansible/blob/master/tests/bootstrap-aio.yml#L28-L46

>As I started to poke, I need to figure out the best way to accomplish
>something. Namely, shade's functional tests expect a clouds.yaml that
>defines two entires - one admin and one non-admin.

>OSA already creates a clouds.yaml with admin creds, so that's done. And
>there is already tempest setup that creates a demo user, so that's done
>too. Here's where I could use some advice:

>- What's the best way to plumb through an option to also write an entry
>to clouds.yaml for the demo user? I'm thinking a boolean in
>openstack-ansible-openstack_openrc like
>"openrc_clouds_yml_add_demo_user" that can be wrapped around a second
>entry in clouds.yaml.j2?

Well, an option could be that an arbitrary dict could be looped through to add 
any content for additional entries in clouds.yml – we could then have the 
tempest demo account settings pulled out of the tempest role (they’re hard 
coded there now, and probably should not be), then have the role loop through 
the same dict if it’s defined. The dict can then be defined in the utility_all 
group vars or are perhaps best only set in the AIO prep config (so that they 
don’t end up being used in production deployments).

>- Can we create the demo user and not run tempest? It's in the
>os_tempest role at the moment. Should I split out the user creation bits
>into a "test_setup" role that tempest depends on - and then also make a
>shade_test role that also depends on the test_setup role?

Well, creating a user and a different clouds.yaml file could also be done with 
a few tasks in a playbook. Unless there’s an opportunity for re-use I don’t 
really see the point of creating a role for it.

>- If we do a test_setup role, should that maybe just write a whole
>additional clouds.yaml out that has the admin and the demo user so that
>we don't have to put conditionals in places that also get used for prod
>deploys?

The existing tempest role creation of a static set of demo accounts is terrible 
anyway. We should ideally make them adaptable and ensure that the passwords are 
randomized.



Rackspace Limited is a company registered in England & Wales (company 
registered number 03897010) whose registered office is at 5 Millington Road, 
Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be 
viewed at www.rackspace.co.uk/legal/privacy-policy - This e-mail message may 
contain confidential or privileged information intended for the recipient. Any 
dissemination, distribution or copying of the enclosed material is prohibited. 
If you receive this transmission in error, please notify us immediately by 
e-mail at ab...@rackspace.com and delete the original message. Your cooperation 
is appreciated.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [openstack-ansible] Adding a scenario for shade's functional tests

[Monty Taylor]

Hey all!

I woke up early this morning and found myself thinking "I clearly don't 
have enough work on my plate, why don't I add some more?"


I'd love to make a gate job that:
- Deploys a cloud with openstack-ansible
- Runs shade's functional tests against that cloud

Which makes me think it just needs to be an additional scenario perhaps?

As I started to poke, I need to figure out the best way to accomplish 
something. Namely, shade's functional tests expect a clouds.yaml that 
defines two entires - one admin and one non-admin.


OSA already creates a clouds.yaml with admin creds, so that's done. And 
there is already tempest setup that creates a demo user, so that's done 
too. Here's where I could use some advice:


- What's the best way to plumb through an option to also write an entry 
to clouds.yaml for the demo user? I'm thinking a boolean in 
openstack-ansible-openstack_openrc like 
"openrc_clouds_yml_add_demo_user" that can be wrapped around a second 
entry in clouds.yaml.j2?


- Can we create the demo user and not run tempest? It's in the 
os_tempest role at the moment. Should I split out the user creation bits 
into a "test_setup" role that tempest depends on - and then also make a 
shade_test role that also depends on the test_setup role?


- If we do a test_setup role, should that maybe just write a whole 
additional clouds.yaml out that has the admin and the demo user so that 
we don't have to put conditionals in places that also get used for prod 
deploys?


I think that's about all the questions I have for now ...

Thanks!
Monty

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev