On Thu, Jun 18, 2015 at 02:13:56PM -0400, David Kranz wrote:
> We had a discussion about this at the qa meeting today around the following
> proposal:
> 
> tl;dr The test accounts feature provides the same functionality as the
> embedded credentials. We should deprecate the account information embedded
> directly in tempest.conf in favor of test-accounts, and remove those options
> at the beginning of the M cycle. We would also rework the non-isolated jobs
> to use parallel test accounts, with and without admin creds. Starting now,
> new features such as cleanup and tempest config will not be required to work
> well (or at all) if the embedded creds are used instead of test accounts.

So this was always the long term plan when we started work on the test-accounts
mechanism about a year ago. We were holding off on deprecating the original
config option based approach until finished the role and network support for
test accounts and we had jobs setup using the mechanism. Now that the we
finished both role and network support all that's left is setting up the jobs.
I don't think there would be any opposition to marking the user and alt user
options as deprecated after that. Also leaving in line comments (and maybe emit
a warning) marking the non-locking provider mechanism as going away, probably
in M. That way we start clearly marking to users that these options will be
going away.

> 
> We have (at least) three use cases that are important, and we want tempest
> to work well with all of them, but that means something different in each
> case:
> 
> 1. throw-away clouds (ci, gate)
> 2. test clouds
> 3. production clouds

Well, tempest is designed to and tries to support running against any OpenStack
cloud. I'm not sure if there are more deployment types than these 3 categories
but we should also be supporting those too.

> 
> For (1), the most important thing is that failing tests not cause false
> negatives in other tests due to re-using a tenant. This makes tenant
> isolation continue to be a good choice here, and requiring admin is not an
> issue. In a perfect world where tempest never left behind any resources
> regardless of an error at any line of code, test accounts could be used. But
> we are probably a long way from that.

So the cleanup issue here is actually a wider openstack issue. Tempest will
*always* call delete on created projects and users. This was a requirement for
making test accounts work. (the mechanism for calling delete or freeing a 
credential set from the list is shared) With tenant isolation this means we'll
be deleting a project and users and resources scoped to either may not be
deleted first. (if there is a tempest or openstack bug) This is a wider issue
for all OpenStack projects that there was a thread a few months ago discussing.

> 
> For (3), we cannot use admin creds for tempest runs, and test accounts with
> cleanup allow parallel execution, accepting the risk of a leak causing a
> false negative. The only way to avoid that risk is to stamp out all leak
> bugs in tempest.

Well depending on the resource in leak in question test accounts would likely
catch the issues if there is a list on that resource in a later test. But, I
agree, resource leaks have always been treated as bugs and we'll continue to
do so.

> 
> For (2), either isolation or test accounts with cleanup can be used
> 
> The tempest.conf values are not used in any of these scenarios. Is there a
> reason they are needed for anything?
> 

So the only thing that uses config options for credentials is actually tenant
isolation, which uses them to provide admin credentials to do the dynamic
creation of accounts. The real advantage of tenant isolation, besides not
reusing anything, is actually its configuration simplicity. Using an accounts
file can be tricky to use, there are a lot of little gotchas and assumptions
in how you write the file. (which we try to document in both the config guide
and the sample accounts.yaml file) It also requires a large number of users to
be provided depending on the concurrency you're running with. While tenant
isolation requires just setting 3-5 config options and you're fine after that.

I don't think requiring the use of an accounts file for tenant isolation makes
much sense, it's really heavyweight for 1 set of admin creds. Which probably
means we should keep the admin user config option around. Although it might make
more sense to move those options to the auth section. (and maybe rename them to
make it clear that it's for tenant isolation only)


-Matt Treinish

Attachment: signature.asc
Description: PGP signature

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to