If the approach would be to use the client tools to "only smoke test" the
Openstack API, I would somewhat
agree. I think the smoke tests should be shallow but wide, covering every API
in OpenStack but not ever 'scenario'.
So to me a "subset of test cases" would be a subset of "test cases from every
OpenStack API" that would make a smoke suite.
If the default client tools are ahead of the Tempest Rest client library with
regard to API coverage, then it would make sense to use them
to quickly build a smoke test suite, IMHO.
The client managers should be used only to contribute to smoke tests, and not
used randomly for more detailed scenario tests.
I'd agree with Daryl in saying that we should avoid Tempest to smoke test or
any test the client tools, rather use them to smoke test
the API. The client tools should have their own tests.
-Rohit
From: openstack-bounces+rohit.karajgi=nttdata....@lists.launchpad.net
[mailto:openstack-bounces+rohit.karajgi=nttdata....@lists.launchpad.net] On
Behalf Of Daryl Walleck
Sent: Friday, May 04, 2012 1:24 AM
To: Jay Pipes
Cc: openstack-qa-t...@lists.launchpad.net; <openstack@lists.launchpad.net>
Subject: Re: [Openstack] [QA] Aligning "smoke" / "acceptance" / "promotion"
test efforts
So my first question is around this. So is the claim is that the client tools
are the default interface for the applications? While that works for coders in
python, what about people using other languages? Even then, there's no
guarantee that the clients in different languages are implemented in the same
way. Tempest was designed originally because while it does use an abstraction
between the API and the tests, there is nothing to "assist" the user by
retrying and the like. While I think there's a place for writing tests using
the command line clients, to me that would be a smoke test of a client and not
as much a smoke test of the API.
Daryl
On May 3, 2012, at 12:01 PM, Jay Pipes wrote:
However, before this can happen, a number of improvements need to be made to
Tempest. The issue with the "smoke tests" in Tempest is that they aren't really
smoke tests. They do not use the default client tools (like novaclient,
keystoneclient, etc) and are not annotated consistently.
______________________________________________________________________
Disclaimer:This email and any attachments are sent in strictest confidence for
the sole use of the addressee and may contain legally privileged, confidential,
and proprietary data. If you are not the intended recipient, please advise the
sender by replying promptly to this email and then delete and destroy this
email and any attachments without any further use, copying or forwarding
_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
