Yes, we have Tempest and unit tests and Functional tests. But still we
need test plans....
Keystone often plays a role deep in others work flows. Often we have
complicated features that span multiple services; WebSSO, Trusts and
HEAT, EC2 Credentials...
Before we can automate the tests, we need to know what to test. And
that is a conversation best started by the developers implementing the
feature, and then handed off to the engineer that needs to make sure the
feature actually works, positive, negative, and edge.
I'm sure that we have many organizations with internal QA that can
benefit from test plans that show how to exercise and test these
features. As part of getting comfortable working up stream, my team has
written the start of a few test plans. I'd like to share them with the
larger Keystone consuming community;
https://etherpad.openstack.org/p/ldap-backend-non-default-domain
https://etherpad.openstack.org/p/hierarchical-projects
https://etherpad.openstack.org/p/keystone-token-scoping
As you can guess, my goal is not purely altruistic: I'd like to get
other people to both input into these test cases and to write some for
other aspects of Keystone. Other ones we've identified:
CADF Notifications Everywhere
Allow Redelegation via Trusts
mapping to existing user in federated workflow
I think Etherpad is the right medium for this. I was originally
thinking the Spec process, but approving test plans seems to
centralized: we want multiple engineers contributing to the test
plans. They should be collaborative documents as people add suggestions
of what to test.
Eventually, these should be automated, but before we can automate, we
need to identify what to test. Getting just that down is incredibly
valuable. We can be more rigorous on the review process when we automate.
I am actively seeking suggestions. How should we best organize this effort?
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
