On 03/16/2016 11:01 PM, Ken'ichi Ohmichi wrote:
2016-03-16 19:29 GMT-07:00 Adam Young <ayo...@redhat.com>:
On 03/16/2016 09:20 PM, Ken'ichi Ohmichi wrote:
Hi
I have one proposal[1] related to negative tests in Tempest, and
hoping opinions before doing that.
Now Tempest contains negative tests and sometimes patches are being
posted for adding more negative tests, but I'd like to propose
removing them from Tempest instead.
Negative tests verify surfaces of REST APIs for each component without
any integrations between components. That doesn't seem integration
tests which are scope of Tempest.
In addition, we need to spend the test operating time on different
component's gate if adding negative tests into Tempest. For example,
we are operating negative tests of Keystone and more
components on the gate of Nova. That is meaningless, so we need to
avoid more negative tests into Tempest now.
If wanting to add negative tests, it is a nice option to implement
these tests on each component repo with Tempest plugin interface. We
can avoid operating negative tests on different component gates and
each component team can decide what negative tests are valuable on the
gate.
Hear hear! For Keystone, please put them in the Keystone Client Functional
tests.
They are negative tests of Keystone API, not keystoneclient tests.
These tests are implemented with Tempest original rest clients without
official clients because it is nice to test variable cases in Tempest.
So it is nice to keep them in Keystone repo, I feel.
That works, too, and there is a Functional test section in Keystone Proper.
Most of the Keystone APIs are fairly well covered by the Unit tests,
which make a full HTTP call through a liteish server, so the functional
tests are really to cover the live database (soon to include LDAP,
hopefully) integrations.
Thanks
Ken Ohmichi
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
