Sean,

I do understand why we have tempest for python-cinderclient now.

But my point is: tempest runs more than 200 tests per each cinderclient
change request which takes a lot of time. Why can't we just introduce few
integration tests which will tests nova<=>python-cinderclient API.

Also, Nova is not only one consumer of cinderclient. What about Heat? We
don't want to break it too but to run all Heat-related Tempest tests is not
a good idea. We have to implement integration tests between Heat and
python-cinderclient too.

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Tue, Feb 16, 2016 at 2:05 PM, Sean Dague <s...@dague.net> wrote:

> On 02/15/2016 02:48 PM, Ivan Kolodyazhny wrote:
> > Hi all,
> >
> > I'll talk mostly about python-cinderclient but the same question could
> > be related for other clients.
> >
> > Now, for python-cinderclient we've got to kinds for
> > functional/integrated jobs:
> >
> > 1) gate-cinderclient-dsvm-functional - a very limited (for now) set of
> > functional tests, most of them were part of tempest CLI tests in the
> past.
> >
> > 2) gate-tempest-dsvm-neutron-src-python-cinderclient - if I understand
> > right, the idea os this job was to have integrated tests to test
> > cinderclient with other projects to verify that new patch to
> > python-cinderclietn won't break any other project.
> > But it does *not* test cinderclient at all, except few attach-related
> > tests because Tempest doesn't use python-*client.
>
> This does test the real world usage of Nova consuming
> python-cinderclient. That's why it's still there. This ensures that a
> cinderclient upcoming release won't completely tank the integrated gate.
> All openstack libraries that get used by all servers in openstack have
> something equivalent.
>
> > The same job was added for python-heatclient but was removed because
> > devstack didn't install Heat for that job [1].
> >
> > We agreed [2] to remove this job from cinderclient gates too, once
> > functional or integration tests will be implemented.
>
> Um, what now?
>
> > There is a proposal to python-cinderclient tests to implement some
> > cross-project testing to make sure, that new python-cinderclient won't
> > break any of existing project who use it.
> >
> > After discussing in IRC with John Griffith (jgriffith) I'm realized that
> > it could be an cross-project initiative in such kind of integration
> > tests. OpenStack Client (OSC) could cover some part of such tests, but
> > does it mean that we'll run OSC tests on every patch to python-*client?
> > We can run only cinder-realated OSC tests on our gates to verify that it
> > doesn't breack OSC and, may be other project.
> >
> > The other option, is to implement tests like [3] per project basis and
> > call it "integration".  Such tests could cover more cases than OSC
> > functional tests and have more project-related test cases, e.g.: test
> > some python-cinderclient specific corner cases, which is not related to
> OSC.
> >
> > IMO, It would be good to have some cross-project decision on how will be
> > implement clients' integration tests per project.
> >
> >
> > [1] https://review.openstack.org/#/c/272411/
> > [2]
> >
> http://eavesdrop.openstack.org/meetings/cinder/2015/cinder.2015-12-16-16.00.log.html
> > [3] https://review.openstack.org/#/c/279432/8
> >
> > Regards,
> > Ivan Kolodyazhny,
> > http://blog.e0ne.info/
> >
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
>
> --
> Sean Dague
> http://dague.net
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to