On 3/21/14 3:49 PM, "Rochelle.RochelleGrober" <rochelle.gro...@huawei.com> wrote:
> >> From: Malini Kamalambal [mailto:malini.kamalam...@rackspace.com] ><snip> >> >> We are talking about different levels of testing, >> >> 1. Unit tests - which everybody agrees should be in the individual >> project >> itself >> 2. System Tests - 'System' referring to (& limited to), all the >> components >> that make up the project. These are also the functional tests for the >> project. >> 3. Integration Tests - This is to verify that the OS components >> interact >> well and don't break other components -Keystone being the most obvious >> example. This is where I see getting the maximum mileage out of >> Tempest. >> >> I see value in projects taking ownership of the System Tests - because >> if >> the project is not 'functionally ready', it is not ready to integrate >> with >> other components of Openstack. >> But for this approach to be successful, projects should have diversity >> in >> the team composition - we need more testers who focus on creating these >> tests. >> This will keep the teams honest in their quality standards. > >+1000 I love your approach to this. You are right. Functional tests >for the project, that exist in an environment, but that exercise the >intricacies of just the project aren't there for most projects, but >really should be. And these tests should be exercised against new code >before the code enters the gerrit/Jenkins stream. But, as Malini points >out, it's at most a dream for most projects as the test developers just >aren't part of most projects. > > >> As long as individual projects cannot guarantee functional test >> coverage, >> we will need more tests in Tempest. >> But that will shift focus away from Integration Testing, which can be >> done >> ONLY in Tempest. > >+1 This is also an important point. If functional testing belonged to >the projects, then most of these tests would be run before a tempest test >was ever run and would not need to be part of the integrated tests, >except as a subset that demonstrate the functioning integration with >other projects. > >> >> Regardless of whatever we end up deciding, it will be good to have >> these >> discussions sooner than later. >> This will help at least the new projects to move in the right >> direction. > >Maybe a summit topic? How do we push functional testing into the project >level development? > >--Rocky That is a good idea I just suggested a session for the ATL summit. See http://summit.openstack.org/cfp/details/134 > >> >> -Malini >> >> >> >> >> >> >> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >_______________________________________________ >OpenStack-dev mailing list >OpenStack-dev@lists.openstack.org >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev