Re: [openstack-dev] [openstack-qa] [qa] tempest

[Frittoli, Andrea (HP Cloud)]

Hi,

please note that the openstack-qa distribution list is not used anywhere, we 
only keep it for periodic tests notifications.
You should use openstack-dev instead.

Documentation is available at [0] and in the sample file [1].
Tempest configuration is uploaded to logs after each CI job - see for instance 
[2].
You can use that to see how tempest is configured for different jobs and/or 
different branches.

You can also activate the tempest service in devstack, and you will get a 
functional tempest.conf in there based on your devstack setup.

hth

andrea

[0] http://docs.openstack.org/developer/tempest
[1] http://git.openstack.org/cgit/openstack/tempest/tree/etc/tempest.conf.sample
[2] 
http://logs.openstack.org/43/206643/2/check/gate-tempest-dsvm-full/37261f2/logs/tempest_conf.txt.gz


On 30 Jul 2015, at 08:58, wh王昊(IT) mailto:w_...@ctrip.com>> 
wrote:


Hi, There is my configuration file:

[DEFAULT]
lock_path=/var/lock/tempest
log_dir=/var/log/tempest

[cli]
enabled=true
cli_dir=/usr/bin
has_manage=false


[compute]
image_ref=b53bf015-ad14-4daf-ae54-496d2b69c1d3
image_ref_alt=b53bf015-ad14-4daf-ae54-496d2b69c1d3

flavor_ref=871a7187-e630-44e4-94a0-96b692e520df
flavor_ref_alt=3a176f86-3cb7-42d5-bc71-6e385172a903

build_timeout=600
run_ssh=false
ssh_auth_method=disabled
ssh_connect_method=floating
ip_version_for_ssh=4
use_floatingip_for_ssh=false
allow_tenant_isolation=True


[compute-admin]
username=admin
tenant_name=admin
password=**


[compute-feature-enabled]
api_v3=false
change_password=false
console_output=false
resize = true
pause = false
shelve=false
live_migration=false
snapshot = false


[dashboard]
dashboard_url=http://10.18.5.149/
login_url=http://10.18.5.149/auth/login/


[identity]
auth_version = v2
admin_domain_name = Default
admin_tenant_id = 8732aeea845543e9a5981e1a6d758512
admin_tenant_name = admin
admin_password = **
admin_username = admin
alt_tenant_name = alt_demo
alt_password = **
alt_username = alt_demo
tenant_name = demo
password = **
username = demo
uri_v3 = http://10.18.5.149:5000/v3/
uri = http://10.18.5.149:5000/v2.0/

[identity-feature-enabled]
trust=true
api_v2=true
api_v3=False


[network]
#default_network = 10.0.0.0/24
#public_router_id =
public_network_id = 663d43bc-461e-4c3e-bed8-384fa2ae8aff
tenant_networks_reachable = false
api_version = 2.0


[network-feature-enabled]
api_extensions = 
allowed-address-pairs,ext-gw-mode,binding,quotas,agent,dhcp_agent_scheduler,provider,external-net,extra_dhcp_opt,l3_agent_scheduler,router,extraroute
#api_extensions=all
ipv6_subnet_attributes = False
ipv6 = False

[service_available]
neutron = True
heat = False
ceilometer = False
swift = False
cinder = False
nova = True
glance = True
horizon = True

you can get image_ref id from horizon or use command line “nova image-list”.

发件人: Gil Halperin [mailto:gihalpe...@advaoptical.com]
发送时间: 2015年7月30日 15:41
收件人: openstack...@lists.openstack.org
主题: [openstack-qa] tempest

Hey, I would like to get help with Tempest, Ive been setting tempest up for 
testing my openstack server, I've been using testr to run the tests but I need 
some help to figure out stuff on the “tempest.config” file, for example, at the 
compute section, what "ID" do i need to write in the "image_ref" row?
is there any other manual or some help docs for tempest other than 
"http://docs.openstack.org/developer/tempest/configuration.html#tempest-configuration";
 ?
any other full explanation of “tempest.config” file?
thanks
___
openstack-qa mailing list
openstack...@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] Proposed Changes to Tempest Core

[Frittoli, Andrea (HP Cloud)]

Thank you everyone for your votes and your trust.
I'm proud to join the tempest core team!

Andrea

Sent from my tiny device


 Matthew Treinish wrote 

So all of the current core team members have voted unanimously in favor of
adding Andrea to the team.

Welcome to the team Andrea.

-Matt Treinish

On Fri, Jul 25, 2014 at 01:32:27PM -0400, Attila Fazekas wrote:
> +1
>
>
> - Original Message -
> > From: "Matthew Treinish" 
> > To: openstack-dev@lists.openstack.org
> > Sent: Tuesday, July 22, 2014 12:34:28 AM
> > Subject: [openstack-dev] [QA] Proposed Changes to Tempest Core
> >
> >
> > Hi Everyone,
> >
> > I would like to propose 2 changes to the Tempest core team:
> >
> > First, I'd like to nominate Andrea Fritolli to the Tempest core team. Over
> > the
> > past cycle Andrea has been steadily become more actively engaged in the
> > Tempest
> > community. Besides his code contributions around refactoring Tempest's
> > authentication and credentials code, he has been providing reviews that have
> > been of consistently high quality that show insight into both the project
> > internals and it's future direction. In addition he has been active in the
> > qa-specs repo both providing reviews and spec proposals, which has been very
> > helpful as we've been adjusting to using the new process. Keeping in mind
> > that
> > becoming a member of the core team is about earning the trust from the
> > members
> > of the current core team through communication and quality reviews, not
> > simply a
> > matter of review numbers, I feel that Andrea will make an excellent addition
> > to
> > the team.
> >
> > As per the usual, if the current Tempest core team members would please vote
> > +1
> > or -1(veto) to the nomination when you get a chance. We'll keep the polls
> > open
> > for 5 days or until everyone has voted.
> >
> > References:
> >
> > https://review.openstack.org/#/q/reviewer:%22Andrea+Frittoli+%22,n,z
> >
> > http://stackalytics.com/?user_id=andrea-frittoli&metric=marks&module=qa-group
> >
> >
> > The second change that I'm proposing today is to remove Giulio Fidente from
> > the
> > core team. He asked to be removed from the core team a few weeks back 
> > because
> > he
> > is no longer able to dedicate the required time to Tempest reviews. So if
> > there
> > are no objections to this I will remove him from the core team in a few 
> > days.
> > Sorry to see you leave the team Giulio...
> >
> >
> > Thanks,
> >
> > Matt Treinish
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] [rfc] move scenario tests to tempest client

[Frittoli, Andrea (HP Cloud)]

++

The ugly monkey patch approach is still working fine for my downstream 
testing, but that's something I'd be happy to get rid of.

Something that may be worth considering is to have an abstraction layer on top 
of tempest clients, to allow switching the actual implementation below:

- REST call as now for the gate  jobs
- python calls for running the tests in non-integrated environments - these 
would live in-tree with the services rather than in tempest - similar  to what 
the neutron team is doing to run tests in tree
- python calls to the official clients, so that a tempest run could still be 
used to verify the python bindings  in a dedicated job

andrea

-Original Message-
From: Sean Dague [mailto:s...@dague.net]
Sent: 10 July 2014 12:23
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [qa] [rfc] move scenario tests to tempest client

As I've been staring at failures in the gate a lot over the past month, we've 
managed to increasingly tune the tempest client for readability and 
debugability. So when something fails in an API test, pin pointing it's 
failure point is getting easier. The scenario tests... not so much.

Using the official clients in the scenario tests was originally thought of as 
a way to get some extra testing on those clients through Tempest.
However it has a ton of debt associated with it. And I think that client 
testing should be done as functional tests in the client trees[1], not as a 
side effect of Tempest.

 * It makes the output of a fail path radically different between the 2 types
 * It adds a bunch of complexity on tenant isolation (and basic duplication 
between building accounts for both clients)
 * It generates a whole bunch of complexity around "waiting for"
resources, and safe creates which garbage collect. All of which has to be done 
above the client level because the official clients don't provide that 
functionality.

In addition the official clients don't do the right thing when hitting API 
rate limits, so are dubious in running on real clouds. There was a proposed 
ugly monkey patch approach which was just too much for us to deal with.

Migrating to tempest clients I think would clean up a ton of complexity, and 
provide for a more straight forward debuggable experience when using Tempest.

I'd like to take a temperature on this though, so comments welcomed.

-Sean

[1] -
http://lists.openstack.org/pipermail/openstack-dev/2014-July/039733.html
(see New Thinking about our validation layers)

--
Sean Dague
http://dague.net



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] Questions about test policy for scenario test

[Frittoli, Andrea (HP Cloud)]

There's a spec in progress related to this,  I'd love to see your comments in 
there:

https://review.openstack.org/#/c/94741/

Andrea

Sent from my tiny device


 Daryl Walleck wrote 

I really like this option, especially if it leaves a generic hook available for 
validation. This could allow for different types of compute validators such as 
hypervisor specific or 3rd party compute validators to be implemented.

From: Fei Long Wang mailto:feil...@catalyst.net.nz>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Date: Wednesday, June 25, 2014 at 5:06 PM
To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [QA] Questions about test policy for scenario test

Good to know. I think it's a good idea to implement a common compute verifier 
after instances booted. Maybe we can define different checking levels so that 
it can be leveraged by different test cases. I will see what I can do.

On 24/06/14 22:27, Sean Dague wrote:

On 06/24/2014 01:29 AM, Fei Long Wang wrote:


Greetings,

We're leveraging the scenario test of Tempest to do the end-to-end
functional test to make sure everything work great after upgrade,
patching, etc. And We're happy to fill the gaps we found. However, I'm a
little bit confused about the test policy from the scenario test
perspective, especially comparing with the API test. IMHO, scenario test
will cover some typical work flows of one specific service or mixed
services, and it would be nice to make sure the function is really
working instead of just checking the object status from OpenStack
perspective. Is that correct?

For example, live migration of Nova, it has been covered in API test of
Tempest (see
https://github.com/openstack/tempest/blob/master/tempest/api/compute/test_live_block_migration.py).
But as you see, it just checks if the instance is Active or not instead
of checking if the instance can be login/ssh successfully. Obviously,
from an real world view, we'd like to check if it's working indeed. So
the question is, should this be improved? If so, the enhanced code
should be in API test, scenario test or any other places? Thanks you.


The fact that computes aren't verified fully during the API testing is
mostly historical. I think they should be. The run_ssh flag used to be
used for this, however because of some long standing race conditions in
the networking stack, that wasn't able to be turned on in upstream
testing. My guess is that it's rotted now.

We've had some conversations in the QA team about a compute verifier
that would be run after any of the compute jobs to make sure they booted
correctly, and more importantly, did a very consistent set of debug
capture when they didn't. Would be great if that's something you'd like
to help out with.

-Sean





___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
Cheers & Best regards,
Fei Long Wang (王飞龙)
--
Senior Cloud Software Engineer
Tel: +64-48032246
Email: flw...@catalyst.net.nz
Catalyst IT Limited
Level 6, Catalyst House, 150 Willis Street, Wellington
--
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] Should Assignee, Milestone really be in the qa-spec?

[Frittoli, Andrea (HP Cloud)]

+1, I would remove them as well, assignee and milestone probably do not need
to go through review, it something we can agree upon at QA meetings.

andrea

-Original Message-Cl@ud3D3bu$$y
From: David Kranz [mailto:dkr...@redhat.com] 
Sent: 11 June 2014 20:18
To: OpenStack Development Mailing List
Subject: [openstack-dev] [qa] Should Assignee, Milestone really be in the
qa-spec?

While reviewing some specs I noticed that I had put myself down for more
Juno-2 work than is likely to be completed. I suspect this will happen
routinely with many folks. Also, assignees may change. This information is
not really part of the spec at all. Since we are still using blueprints to
actually track progress, I think it would be better to use the corresponding
fields in blueprints to make sure these values reflect reality on an ongoing
basis.

  -David

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] Tests for Custom roles in keystone v3

[Frittoli, Andrea (HP Cloud)]

Hi Ajaya,

 

Thanks for offer to help :)

 

Are you talking about tempest tests or in-tree keystone tests?

 

Verifying custom roles can be challenging via  API only driven tests such as 
tempest – as it requires to have the policies configured accordingly in the 
cloud under test (i.e. devstack).

It should be possible to prepare support for custom roles in the policy at 
deployment time. If tempest what you’re aiming at, it would be good if you 
could file a bp to describe what kind of use cases you have in mind, and why 
you’d like to run them in tempest.

 

As these would be keystone only tests, I wonder if they would be more fitting 
as unit / functional tests in the keystone tree? This approach would give you 
more flexibility in changing the policies.

 

If you are interested in contributing to tempest tests in the keystone area, 
below are some ideas.

 

A few bp which are related to tempest and keystone identity API v3:

-  Refactor tempest so that it may run consuming identity v3 only (or 
greater, when available) [1]

-  Setup dsvm tests which rely on identity v3 only (including 
intra-service communication) [2]

-  Cross domain testing: write tests to verify the impact of the domain 
scope on keystone itself and on the services [3]

-  Tempest without admin account (David Kranz’s blueprint): run tempest 
tests without the need of an “admin” account [4]

 

Your very welcome to contribute to any of those. [3] and [4] are still in the 
design phase.

 

The non-admin blueprint is loosely related to custom roles: it raised the 
question of how to run as many tests as possible without the need of an 
identity-admin account, which in certain deployments may be not available to 
the person running the tests.

The concept of domain introduced in identity v3 may be helpful here, as a 
domain admin could be able to have full control within the boundaries of the 
domain.  

That can be true for keystone, as long as roles is defined and the policy in 
keystone is configured correctly.  

 

For services I believe there is no combination of custom roles / service 
policies that will allow to achieve this – to make an example use case, allow 
the domain admin to list all the VMs, images, containers and networks defined 
within projects that belong to the domain. I believe that for this to be 
possible we’ll have to wait for the hierarchical multi-tenancy in every 
projects. [5]

 

Andrea

 

p.s.

Please use the openstack-dev list, openstack-qa is only used for reporting of 
periodic job test results. 

 

[1] 
https://github.com/openstack/qa-specs/blob/master/specs/multi-keystone-api-version-tests.rst

[2] https://github.com/openstack/qa-specs/blob/master/specs/keystone-v3-jobs.rst

[3] 
http://docs-draft.openstack.org/98/83898/5/check/gate-qa-specs-docs/4372c5f/doc/build/html/specs/cross-domain-testing.html
 

[4] 
http://docs-draft.openstack.org/67/86967/6/check/gate-qa-specs-docs/d0c8170/doc/build/html/specs/run-without-admin.html
 

[5] 
https://etherpad.openstack.org/p/juno-cross-project-hierarchical-multitenancy 

 

From: Ajaya Agrawal [mailto:ajku@gmail.com] 
Sent: 03 June 2014 20:38
To: openstack-qa
Subject: [openstack-qa] Tests for Custom roles in keystone v3

 

Hi,

 

Is someone writing tests for custom roles and policies in keystone v3. for e.g. 
one could create a role called project_admin who would allowed to create/delete 
users in his project only.

 

Andrea, Sean said in irc that you are working on this thing. Would you like to 
have one more pair of hands on this? :)




Cheers,

Ajaya



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa][nova] Status of v3 tests in tempest

[Frittoli, Andrea (HP Cloud)]

Thanks for bringing this up.

We won't be testing v3 in Juno, but we'll need coverage for v2.1.

In my understanding will be a v2 compatible API - so including proxy to
glance cinder and neutron - but with micro-versions to bring in v3 features
such as CamelCase and Tasks.
So we should be able to reuse a good chunk of the v3 test code for testing
v2.1. Adding some config options for the v2.1 to v3 differences we could try
and use the same tests for icehouse v3 and juno v2.1.

We may have to implement support for micro-versions in tempests own rest
client as well.

andrea


-Original Message-
From: David Kranz [mailto:dkr...@redhat.com] 
Sent: 19 May 2014 10:49
To: OpenStack Development Mailing List
Subject: [openstack-dev] [qa][nova] Status of v3 tests in tempest

It seems the nova team decided in Atlanta that "v3" as currently understood
is never going to exist:
https://etherpad.openstack.org/p/juno-nova-v3-api.

There are a number of patches in flight that tweak how we handle supporting
both v2/v3 in tempest to reduce duplication.
We need to decide what to do about this. At a minimum, I think we should
stop any work that is inspired by any v3-related activity except to revert
any v2/v3 integration that was already done. We should really rip out the v3
stuff that was recently added. I know Matt had some concern about that
regarding testing v3 in stable/icehouse but perhaps he can say more.

  -David

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Hierarchical administrative boundary [keystone]

[Frittoli, Andrea (HP Cloud)]

From: Adam Young [mailto:ayo...@redhat.com] 
Sent: 09 May 2014 04:19
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] Hierarchical administrative boundary [keystone]

 

On 05/08/2014 07:55 PM, Tiwari, Arvind wrote:

Hi All,

 

Below is my proposal to address VPC use case using hierarchical
administrative boundary. This topic is scheduled in Hierarchical
Multitenancy
  session of Atlanta design summit.

 

https://wiki.openstack.org/wiki/Hierarchical_administrative_boundary

 

Please take a look.

 

Thanks,

Arvind

 






___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org 

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Looks very good.  One question:  Why hierarchical domains and not Projects.
I'm not disagreeing, mind you, just that I think the Nova team is going for
hierarchical Projects. 

 

  _  

Looks good, thank you!

 

But for this to be even more interesting nova (and other services) should be
domain aware - e.g. so that a domain admin could have control on all
resources which belong to users and projects in that domain.

 

andrea

 



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [QA] Open topics for summit in Atlanta

[Frittoli, Andrea (HP Cloud)]

I started an etherpad for people to put down QA topics we could discuss during 
the summit at lunch or over a beer 

https://etherpad.openstack.org/p/juno-summit-open-topics 

 

Feel free to add anything that needs some good face to face brainstorming :)

 

andrea

 

-- 

Andrea Frittoli

QA Tech Lead

HP Cloud ☁

PC Phone: +445601090317

 



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Consuming keystoneclient's Session object in novaclient

[Frittoli, Andrea (HP Cloud)]

Is there a bp to coordinate work on aligning all clients to the Session object?

 

Having a consistent implementation would make users and developers life so much 
easier – not to mention QA :)

 

andrea

 

From: Joe Gordon [mailto:joe.gord...@gmail.com] 
Sent: 08 May 2014 21:55
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova] Consuming keystoneclient's Session object 
in novaclient

 

 

 

On Wed, May 7, 2014 at 7:54 PM, Jamie Lennox mailto:jamielen...@redhat.com> > wrote:



- Original Message -
> From: "Monty Taylor" mailto:mord...@inaugust.com> >
> To: openstack-dev@lists.openstack.org 
>  
> Sent: Thursday, May 8, 2014 8:22:21 AM
> Subject: Re: [openstack-dev] [nova] Consuming keystoneclient's Session object 
> in novaclient
>
> On 05/07/2014 03:10 PM, Joe Gordon wrote:
> >
> >
> >
> > On Tue, May 6, 2014 at 3:22 PM, Jamie Lennox  >  
> >  >> wrote:
> >
> > All,
> >
> > TL;DR: novaclient should be able to use the common transport/auth
> > layers of keystoneclient. If it does there are going to be functions
> > like client.authenticate() that won't operate the same way when a
> > session object is passed. For most users who just use the CRUD
> > operations there will be no difference.
> >
> >
> > I'm hoping that at least some of the nova community has heard of the
> > push for using keystoneclient's Session object across all the
> > clients. For those unaware keystoneclient.session.Session is a
> > common transport and authentication layer to remove the need for
> > each python-*client having there own authentication configuration
> > and disparate transport options.
> >
> > It offers:
> >   - a single place for updates to transport (eg fixing TLS or other
> > transport issues in one place)
> >   - a way for all clients to immediately support the full range of
> > keystone's authentication including v3 auth, SAML, kerberos etc
> >   - a common place to handle version discovery such that we support
> > multiple version endpoints from the same service catalog endpoint.
> >
> > For information of how to interact with a session you can see:
> > http://www.jamielennox.net/blog/2014/02/24/client-session-objects/
> > This mentions the code is uncommitted however has since been
> > committed with a few small details around parameter names being
> > changed. The theory remains the same.
> >
> >
> > To integrate this into novaclient means that if a session= object is
> > passed then the standard HTTPClient code will be ignored in favour
> > of using what was passed. This means that there are changes in the
> > API of the client. In keystoneclient we have take the opinion that
> > by passing a session object then you opt-in to the newer API and
> > therefore accept that some functions are no longer available. For
> > example client.authenticate() is no longer allowed because
> > authentication is not the client's responsibility. It will have no
> > impact on the standard novaclient CRUD operations and so be
> > un-noticed by the vast majority of users.
> >
> > The review showing these changes is here:
> > https://review.openstack.org/#/c/85920
> >
> > To enable this there are a series of test changes to mock client
> > requests at the HTTP layer rather than in the client. This means
> > that we can test that all client operations against the new and old
> > client construction methods and ensure the same requests are being
> > sent. The foundation of this to turn tests into fixtures can be
> > found by following:
> > 
> > https://blueprints.launchpad.net/python-novaclient/+spec/httpretty-testing
> >
> > IMO making these tests into fixtures is a good idea anyway, however
> > I am only pursuing it so that we can transition to using a common
> > Session.
> >
> > Regarding dependencies, novaclient will need a test-requirements.txt
> > on keystoneclient so that it can construct Session objects to test
> > with but it should not need a requirements.txt as the session object
> > is constructed by the user of the client (eg openstackclient,
> > horizon etc).
> >
> >
> > Can we make novaclient use keystoneclient's session by default? And just
> > add this to requirements.
>
> ++
>
> Once it's supported, I would think that someone wanting to use
> novaclient _without_ keystoneclient should be seen as the exception case
> and not the normal case.

So keystoneclient's session is designed to be passed around, rather than 
constructed individually by the clients, so that the same authentication 
mechanisms can be shared by multiple instances of a client. A made up, but not 
unrealistic example:

auth = keystoneclient.auth.identity

Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

[Frittoli, Andrea (HP Cloud)]

I will arrive Sunday late.

If you meet on Monday I’ll see you there ^_^

 

From: Miguel Lavalle [mailto:mig...@mlavalle.com] 
Sent: 01 May 2014 17:28
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

 

I arrive Sunday at 3:30pm. Either Sunday or Monday are fine with me. Lookging 
forward to it :-)

 

On Wed, Apr 30, 2014 at 5:11 AM, Koderer, Marc mailto:m.kode...@telekom.de> > wrote:

Hi folks,

last time we met one day before the Summit started for a short meet-up.
Should we do the same this time?

I will arrive Saturday to recover from the jet lag ;) So Sunday 11th would be 
fine for me.

Regards,
Marc
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org  
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [QA][keystone] Best practices for admin roles and policies

[Frittoli, Andrea (HP Cloud)]

In the QA meeting yesterday, we discussed about accounts, admin roles and 
policies, and how we use them in tempest and in our test environments (e.g. 
devstack and toci).

The conversation was triggered by a discussion on running tempest without admin 
credentials – see [0], [1].

 

So we are looking for recommendations and best practises available on how to 
setup roles in keystone and policies when running keystone v3.

 

A little more background on this.

 

Tempest uses admin accounts for two purposes: 

-  setting up test accounts on the fly, useful to run tests in parallel in 
isolation 

- run tests which require service specific admin privileges, such as list all 
VMs in a cloud for nova, or manage public routers and networks in neutron

 

When running tests with keystone v2, we use an identity admin account, which is 
also acts admin for all other services – only exception being swift, for which 
a dedicated admin role is defined.

We need to define how we want the accounts and roles setup to look like with 
keystone v3. 

 

Keystone V3 provides (at least) two level of admin role: overall identity admin 
and domain admin. 

 

Domain admin is sufficient to create users and tenants within a certain domain, 
which is nice as it could allow tempest to run with a domain admin account only 
and still use tenant isolation.

But how does that map to the service admin roles, given the fact that services 
are not domain aware?

 

For instance a nova list --all-tenants will show all VMs in the cloud, and 
there is no option to see only the VMs owned by users in a certain domain.

Thus the administrator of a single domain should not be able to assign a system 
wide role (such as nova admin) to one user in his/her domain, as it would give 
such user visibility to all the VMs in all domains.

 

[0]https://blueprints.launchpad.net/tempest/+spec/run-without-admin / 
  

[1] https://review.openstack.org/#/c/86967/

 

-- 

Andrea Frittoli

QA Tech Lead

HP Cloud ☁



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] The future of nosetests with Tempest

[Frittoli, Andrea (HP Cloud)]

This is another example of achieving the same result (exclusion from a
list):
https://git.openstack.org/cgit/openstack/tripleo-image-elements/tree/element
s/tempest/tests2skip.py
https://git.openstack.org/cgit/openstack/tripleo-image-elements/tree/element
s/tempest/tests2skip.txt

andrea

-Original Message-
From: Matthew Treinish [mailto:mtrein...@kortar.org] 
Sent: 27 February 2014 15:49
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [QA] The future of nosetests with Tempest

On Tue, Feb 25, 2014 at 07:46:23PM -0600, Matt Riedemann wrote:
> 
> 
> On 2/12/2014 1:57 PM, Matthew Treinish wrote:
> >On Wed, Feb 12, 2014 at 11:32:39AM -0700, Matt Riedemann wrote:
> >>
> >>
> >>On 1/17/2014 8:34 AM, Matthew Treinish wrote:
> >>>On Fri, Jan 17, 2014 at 08:32:19AM -0500, David Kranz wrote:
> On 01/16/2014 10:56 PM, Matthew Treinish wrote:
> >Hi everyone,
> >
> >With some recent changes made to Tempest compatibility with 
> >nosetests is going away. We've started using newer features that 
> >nose just doesn't support. One example of this is that we've 
> >started using testscenarios and we're planning to do this in more
places moving forward.
> >
> >So at Icehouse-3 I'm planning to push the patch out to remove 
> >nosetests from the requirements list and all the workarounds and 
> >references to nose will be pulled out of the tree. Tempest will 
> >also start raising an unsupported exception when you try to run 
> >it with nose so that there isn't any confusion on this moving 
> >forward. We talked about doing this at summit briefly and I've 
> >brought it up a couple of times before, but I believe it is time 
> >to do this now. I feel for tempest to move forward we need to do this
now so that there isn't any ambiguity as we add even more features and new
types of testing.
> I'm with you up to here.
> >
> >Now, this will have implications for people running tempest with 
> >python 2.6 since up until now we've set nosetests. There is a 
> >workaround for getting tempest to run with python 2.6 and testr see:
> >
> >https://review.openstack.org/#/c/59007/1/README.rst
> >
> >but essentially this means that when nose is marked as 
> >unsupported on tempest python 2.6 will also be unsupported by 
> >Tempest. (which honestly it basically has been for while now just 
> >we've gone without making it official)
> The way we handle different runners/os can be categorized as 
> "tested in gate", "unsupported" (should work, possibly some hacks 
> needed), and "hostile". At present, both nose and py2.6 I would 
> say are in the unsupported category. The title of this message and 
> the content up to here says we are moving nose to the hostile 
> category. With only 2 months to feature freeze I see no 
> justification in moving
> py2.6 to the hostile category. I don't see what new testing 
> features scheduled for the next two months will be enabled by 
> saying that tempest cannot and will not run on 2.6. It has been 
> agreed I think by all projects that py2.6 will be dropped in J. It 
> is OK that py2.6 will require some hacks to work and if in the 
> next few months it needs a few more then that is ok. If I am 
> missing another connection between the py2.6 and nose issues, please
explain.
> 
> >>>
> >>>So honestly we're already at this point in tempest. Nose really 
> >>>just doesn't work with tempest, and we're adding more features to 
> >>>tempest, your negative test generator being one of them, that 
> >>>interfere further with nose. I've seen several
> >>
> >>I disagree here, my team is running Tempest API, CLI and scenario 
> >>tests every day with nose on RHEL 6 with minimal issues.  I had to 
> >>workaround the negative test discovery by simply sed'ing that out of 
> >>the tests before running it, but that's acceptable to me until we 
> >>can start testing on RHEL 7.  Otherwise I'm completely OK with 
> >>saying py26 isn't really supported and isn't used in the gate, and 
> >>it's a buyer beware situation to make it work, which includes 
> >>pushing up trivial patches to make it work (which I did a few of 
> >>last week, and they were small syntax changes or usages of 
> >>testtools).
> >>
> >>I don't understand how the core projects can be running unit tests 
> >>in the gate on py26 but our functional integration project is going 
> >>to actively go out and make it harder to run Tempest with py26, that 
> >>sucks.
> >>
> >>If we really want to move the test project away from py26, let's 
> >>make the concerted effort to get the core projects to move with it.
> >
> >So as I said before the python 2.6 story for tempest remains the same 
> >after this change. The only thing that we'll be doing is actively 
> >preventing nose from working with tempest.
> >
> >>
> >>And FWIW, I tried the discover.py

Re: [openstack-dev] [qa] Does scenario.test_minimum_basic need to upload ami images?

[Frittoli, Andrea (HP Cloud)]

Thanks David, +++ 

This is a strong dependency to devstack, and it would be nice if we could
lose it.

andrea

-Original Message-
From: David Kranz [mailto:dkr...@redhat.com] 
Sent: 20 February 2014 21:32
To: OpenStack Development Mailing List
Subject: [openstack-dev] [qa] Does scenario.test_minimum_basic need to
upload ami images?

Running this test in tempest requires an ami image triple to be on the disk
where tempest is running in order for the test to upload it. It would be a
lot easier if this test could use a simple image file instead. That image
file could even be obtained from the cloud being tested while configuring
tempest. Is there a reason to keep the three-part image?

  -David

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] Service dependency decorators in tests

[Frittoli, Andrea (HP Cloud)]

Hi Boris,

 

Matt leads this effort - the bp is
https://blueprints.launchpad.net/tempest/+spec/add-service-tags 

 

andrea

 

From: Boris Pavlovic [mailto:bpavlo...@mirantis.com] 
Sent: 18 February 2014 17:18
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [QA] Service dependency decorators in tests

 

Hi, 

 

 

I will be glad to help with this part! It shouldn't be too much work to
handle this. 

 

Who will lead this thing?) 

 

 

Best regards,

Boris Pavlovic 

 

 

On Tue, Feb 18, 2014 at 7:51 PM, Matthew Treinish mailto:mtrein...@kortar.org> > wrote:

On Tue, Feb 18, 2014 at 09:42:43AM -0500, Sean Dague wrote:
> I'm +1 on that. Mostly it's just a lot of time, so hasn't been dealt
> with yet. Unless there is a completely pressing need, I'd rather see
> that happen right after icehouse release, because I'm concerned it will
> be a lot of changes coming in when people are trying to get other more
> critical things landed.

Yeah I agree with this too, however we said the same thing after havana
regarding service tags.


>
>   -Sean
>
> On 02/18/2014 09:33 AM, Frittoli, Andrea (Cloud Services) wrote:
> > Hi all,
> >
> >
> >
> > Scenario tests feature service dependency decorators in tests - so that
> > a test will run only if all required components are available.
> >
> > I think we should extend them to all tests, including the API ones. For
> > instance Nova image tests depend on Glance, cinder attach/detach tests
> > depend on Nova.

So originally the service tag decorator just added an attr and the intent
was
to just make it easy to filter by service. So, we only were adding the
decorator
to tests that touched the service that weren't in the namespace.
For example:

tests in: api.compute.image would not get an image tag

but if a test in: api.compute.volume touched glance then it should have the
decorator.

But, it was only recently that I added the skip function to the decorator
because it made sense to. I still think we should follow the same convention
for using the services decorators, even with skips as part of the decorator.
For the tests in directories that contain a service name we should raise
skips
for that service in the explicity. Since the directory calls out the service
it
should be implied that it's required so we don't need to bother tagging all
the
tests.


> >
> >
> >
> > If there is agreement on that I'd happy to start a bp to track the test
> > tagging effort.

There has been one open since havana :)
https://blueprints.launchpad.net/tempest/+spec/add-service-tags

-Matt Treinish



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org 

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone][all] Keystone V2 and V3 support in icehouse

[Frittoli, Andrea (HP Cloud)]

Hi, 

 

thanks for the update

 

Link to the tempest bp I’m working on: 
https://blueprints.launchpad.net/tempest/+spec/multi-keystone-api-version-tests 

 

The update of the python binding to use the keystone binding is targeted for 
icehouse or juno? 

 

andrea

 

 

From: Dolph Mathews [mailto:dolph.math...@gmail.com] 
Sent: 18 February 2014 13:54
To: OpenStack Development Mailing List (not for usage questions)
Cc: Jamie Lennox
Subject: Re: [openstack-dev] [keystone][all] Keystone V2 and V3 support in 
icehouse

 

 

On Mon, Feb 10, 2014 at 5:23 PM, Frittoli, Andrea (Cloud Services) 
mailto:fritt...@hp.com> > wrote:

Hi,

 

I’m working on a tempest blueprint to make tempest able to run 100% on keystone 
v3 (or later versions) – the auth version to be used will be available via a 
configuration switch.

 

The rationale is that Keystone V2 is deprecated in icehouse, V3 being the 
primary version. Thus it would be good to have (at least) one of the  gate jobs 
running entirely with keystone v3.

 

Much appreciated! Have a link to that bp?

 

 

There are other components beyond tempest that would need some changes to make 
this happen.

 

Nova and cinder python bindings work only with keystone v2 [0], and as far as I 
know all core services work with keystone v2 (at least by default). 

Is there a plan to support identity v3 there until the end of icehouse?

 

Yes (but maybe not by the end of icehouse)- we'd like to make all other client 
libraries depend on keystoneclient's library for authentication in the long 
run. Jamie Lennox has done a ton of great work to prepare keystoneclient for 
that responsibility during Icehouse.

 

If not I think we may have to consider still supporting v2 in icehouse.

 

v2 should certainly be supported in icehouse; which version other projects 
default to is up to them, but I'd like to see *all* projects at least 
defaulting to v3 by the end of Juno.

 

 

Andrea

 

[0] https://bugs.launchpad.net/python-novaclient/+bug/1262843 

 

-- 

Andrea Frittoli

IaaS Systems Engineering Team 

HP Cloud ☁


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org  
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev