Eric, For now, we test Cinder API with some concurrency only with Rally, so, IMO, it's reasonable get more scenarios for API races fixes.
It's not a hard task to implement new scenarios, they are pretty simple: [11] and [12] [11] https://github.com/openstack/rally/blob/master/rally/plugins/openstack/scenarios/cinder/volumes.py#L535 [12] https://github.com/openstack/rally/blob/master/rally-jobs/cinder.yaml#L516 Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ On Wed, Mar 2, 2016 at 4:50 PM, Eric Harney <ehar...@redhat.com> wrote: > On 03/02/2016 09:36 AM, Ivan Kolodyazhny wrote: > > Eric, > > > > There are Gorka's patches [10] to remove API Races > > > > > > [10] > > > https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified > > > > Regards, > > Ivan Kolodyazhny, > > http://blog.e0ne.info/ > > > > So the second part of my question is, is writing a Rally job to prove > out that code a reasonable task? > > How hard is that to do and what does it look like? > > > On Wed, Mar 2, 2016 at 4:27 PM, Eric Harney <ehar...@redhat.com> wrote: > > > >> On 03/02/2016 06:25 AM, Ivan Kolodyazhny wrote: > >>> Hi Team, > >>> > >>> Here are my thoughts and proposals how to make Cinder testing process > >>> better. I won't cover "3rd party CI's" topic here. I will share my > >> opinion > >>> about current and feature jobs. > >>> > >>> > >>> Unit-tests > >>> > >>> - Long-running tests. I hope, everybody will agree that unit-tests > >> must > >>> be quite simple and very fast. Unit tests which takes more than 3-5 > >> seconds > >>> should be refactored and/or moved to 'integration' tests. > >>> Thanks to Tom Barron for several fixes like [1]. IMO, we it would be > >>> good to have some hacking checks to prevent such issues in a future. > >>> > >>> - Tests coverage. We don't check it in an automatic way on gates. > >>> Usually, we require to add some unit-tests during code review > >> process. Why > >>> can't we add coverage job to our CI and do not merge new patches, > with > >>> will decrease tests coverage rate? Maybe, such job could be voting > in > >> a > >>> future to not ignore it. For now, there is not simple way to check > >> coverage > >>> because 'tox -e cover' output is not useful [2]. > >>> > >>> > >>> Functional tests for Cinder > >>> > >>> We introduced some functional tests last month [3]. Here is a patch to > >>> infra to add new job [4]. Because these tests were moved from > >> unit-tests, I > >>> think we're OK to make this job voting. Such tests should not be a > >>> replacement for Tempest. They even could tests Cinder with Fake Driver > to > >>> make it faster and not related on storage backends issues. > >>> > >>> > >>> Tempest in-tree tests > >>> > >>> Sean started work on it [5] and I think it's a good idea to get them in > >>> Cinder repo to run them on Tempest jobs and 3-rd party CIs against a > real > >>> backend. > >>> > >>> > >>> Functional tests for python-brick-cinderclient-ext > >>> > >>> There are patches that introduces functional tests [6] and new job [7]. > >>> > >>> > >>> Functional tests for python-cinderclient > >>> > >>> We've got a very limited set of such tests and non-voting job. IMO, we > >> can > >>> run them even with Cinder Fake Driver to make them not depended on a > >>> storage backend and make it faster. I believe, we can make this job > >> voting > >>> soon. Also, we need more contributors to this kind of tests. > >>> > >>> > >>> Integrated tests for python-cinderclient > >>> > >>> We need such tests to make sure that we won't break Nova, Heat or other > >>> python-cinderclient consumers with a next merged patch. There is a > thread > >>> in openstack-dev ML about such tests [8] and proposal [9] to introduce > >> them > >>> to python-cinderclient. > >>> > >>> > >>> Rally tests > >>> > >>> IMO, it would be good to have new Rally scenarios for every patches > like > >>> 'improves performance', 'fixes concurrency issues', etc. Even if we as > a > >>> Cinder community don't have enough time to implement them, we have to > ask > >>> for them in reviews, openstack-dev ML, file Rally bugs and blueprints > if > >>> needed. > >>> > >> > >> Are there any recent examples of a fix like this recently where it would > >> seem like a reasonable task to write a Rally scenario along with the > patch? > >> > >> Not being very familiar with Rally (as I think most of us aren't), I'm > >> having a hard time picturing this. > >> > >>> > >>> [1] https://review.openstack.org/#/c/282861/ > >>> [2] http://paste.openstack.org/show/488925/ > >>> [3] https://review.openstack.org/#/c/267801/ > >>> [4] https://review.openstack.org/#/c/287115/ > >>> [5] https://review.openstack.org/#/c/274471/ > >>> [6] https://review.openstack.org/#/c/265811/ > >>> [7] https://review.openstack.org/#/c/265925/ > >>> [8] > >>> > >> > http://lists.openstack.org/pipermail/openstack-dev/2016-March/088027.html > >>> [9] https://review.openstack.org/#/c/279432/ > >>> > >>> > >>> Regards, > >>> Ivan Kolodyazhny, > >>> http://blog.e0ne.info/ > >>> > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
