On Fri, Oct 18, 2013 at 10:34 AM, Steven Hardy <sha...@redhat.com> wrote:
> Hi all, > > Starting a thread to discuss $subject, as requested in: > > https://review.openstack.org/#/c/51558/ > > First a bit of background. I wrote a keystoneclient patch, and ayoung > stated he'd like it tested via tempest before he'd ack it: > > https://review.openstack.org/#/c/48462/ > > So I spoke to ayoung and dkranz on IRC, showing them my local tests for the > patch. dkranz suggested creating a "client_lib" directory, where we could > build out a more comprehensive set of tests over time, adding to the inital > tests related to keystone trusts client additions. > > A couple of things to note: > - These are end-to-end tests, designed to test not only the client, but > also the API and keystone backend functionality. So arguably this could > just be a scenario test, e.g scenario/keystone/test_v3_auth.py > I'd love to be able to run these tests against a wider variety of service configurations (e.g. LDAP!), which tempest is obviously more suitable for. > > - The intention is to excercise logic which is hard to fully test with > unit or integration tests, and to catch issues like incompatibility > between client and API - e.g keystoneclient tests may pass, but we need > to make sure the client actually works against the real keystone API. > All of our tests under keystone.tests.test_keystoneclient fall into this category as well: https://github.com/openstack/keystone/blob/a0e26c1882d83989bee3726a5ae08cbe3f32a2b5/keystone/tests/test_keystoneclient.py https://github.com/openstack/keystone/blob/a0e26c1882d83989bee3726a5ae08cbe3f32a2b5/keystone/tests/test_keystoneclient_sql.py > > Working on Heat has given me a pretty good insight into the python-*client > API's, as we use them to orchestrate actions with every openstack service; > IMO anything we can do to make these interfaces more robust (and catch > bugs, several of which I found already while writing these tests) is a > good-thing (tm). > ++ > > I'd welcome feedback on the patch above, and what will be the most > acceptable approach to the tempest team for adding these tests. > > More links: > > https://review.openstack.org/#/c/51559/ > https://review.openstack.org/#/c/51560/ > https://blueprints.launchpad.net/tempest/+spec/keystoneclient-api > > Thanks! > > Steve > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- -Dolph
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev