Sean, I do understand why we have tempest for python-cinderclient now.
But my point is: tempest runs more than 200 tests per each cinderclient change request which takes a lot of time. Why can't we just introduce few integration tests which will tests nova<=>python-cinderclient API. Also, Nova is not only one consumer of cinderclient. What about Heat? We don't want to break it too but to run all Heat-related Tempest tests is not a good idea. We have to implement integration tests between Heat and python-cinderclient too. Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ On Tue, Feb 16, 2016 at 2:05 PM, Sean Dague <s...@dague.net> wrote: > On 02/15/2016 02:48 PM, Ivan Kolodyazhny wrote: > > Hi all, > > > > I'll talk mostly about python-cinderclient but the same question could > > be related for other clients. > > > > Now, for python-cinderclient we've got to kinds for > > functional/integrated jobs: > > > > 1) gate-cinderclient-dsvm-functional - a very limited (for now) set of > > functional tests, most of them were part of tempest CLI tests in the > past. > > > > 2) gate-tempest-dsvm-neutron-src-python-cinderclient - if I understand > > right, the idea os this job was to have integrated tests to test > > cinderclient with other projects to verify that new patch to > > python-cinderclietn won't break any other project. > > But it does *not* test cinderclient at all, except few attach-related > > tests because Tempest doesn't use python-*client. > > This does test the real world usage of Nova consuming > python-cinderclient. That's why it's still there. This ensures that a > cinderclient upcoming release won't completely tank the integrated gate. > All openstack libraries that get used by all servers in openstack have > something equivalent. > > > The same job was added for python-heatclient but was removed because > > devstack didn't install Heat for that job [1]. > > > > We agreed [2] to remove this job from cinderclient gates too, once > > functional or integration tests will be implemented. > > Um, what now? > > > There is a proposal to python-cinderclient tests to implement some > > cross-project testing to make sure, that new python-cinderclient won't > > break any of existing project who use it. > > > > After discussing in IRC with John Griffith (jgriffith) I'm realized that > > it could be an cross-project initiative in such kind of integration > > tests. OpenStack Client (OSC) could cover some part of such tests, but > > does it mean that we'll run OSC tests on every patch to python-*client? > > We can run only cinder-realated OSC tests on our gates to verify that it > > doesn't breack OSC and, may be other project. > > > > The other option, is to implement tests like [3] per project basis and > > call it "integration". Such tests could cover more cases than OSC > > functional tests and have more project-related test cases, e.g.: test > > some python-cinderclient specific corner cases, which is not related to > OSC. > > > > IMO, It would be good to have some cross-project decision on how will be > > implement clients' integration tests per project. > > > > > > [1] https://review.openstack.org/#/c/272411/ > > [2] > > > http://eavesdrop.openstack.org/meetings/cinder/2015/cinder.2015-12-16-16.00.log.html > > [3] https://review.openstack.org/#/c/279432/8 > > > > Regards, > > Ivan Kolodyazhny, > > http://blog.e0ne.info/ > > > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > -- > Sean Dague > http://dague.net > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev