I'll try to implement such scenario and step-by-step guideline soon. Regards, Ivan Kolodyazhny, http://blog.e0ne.info/
On Wed, Mar 2, 2016 at 5:16 PM, Eric Harney <[email protected]> wrote: > On 03/02/2016 10:07 AM, Ivan Kolodyazhny wrote: > > Eric, > > > > For now, we test Cinder API with some concurrency only with Rally, so, > IMO, > > it's reasonable get more scenarios for API races fixes. > > > > It's not a hard task to implement new scenarios, they are pretty simple: > > [11] and [12] > > > > Sure, these are simple, but I think it's nowhere near that simple to > write a scenario which will prove that "remove API races" works correctly. > > > [11] > > > https://github.com/openstack/rally/blob/master/rally/plugins/openstack/scenarios/cinder/volumes.py#L535 > > [12] > > > https://github.com/openstack/rally/blob/master/rally-jobs/cinder.yaml#L516 > > > > Regards, > > Ivan Kolodyazhny, > > http://blog.e0ne.info/ > > > > On Wed, Mar 2, 2016 at 4:50 PM, Eric Harney <[email protected]> wrote: > > > >> On 03/02/2016 09:36 AM, Ivan Kolodyazhny wrote: > >>> Eric, > >>> > >>> There are Gorka's patches [10] to remove API Races > >>> > >>> > >>> [10] > >>> > >> > https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified > >>> > >>> Regards, > >>> Ivan Kolodyazhny, > >>> http://blog.e0ne.info/ > >>> > >> > >> So the second part of my question is, is writing a Rally job to prove > >> out that code a reasonable task? > >> > >> How hard is that to do and what does it look like? > >> > >>> On Wed, Mar 2, 2016 at 4:27 PM, Eric Harney <[email protected]> > wrote: > >>> > >>>> On 03/02/2016 06:25 AM, Ivan Kolodyazhny wrote: > >>>>> Hi Team, > >>>>> > >>>>> Here are my thoughts and proposals how to make Cinder testing process > >>>>> better. I won't cover "3rd party CI's" topic here. I will share my > >>>> opinion > >>>>> about current and feature jobs. > >>>>> > >>>>> > >>>>> Unit-tests > >>>>> > >>>>> - Long-running tests. I hope, everybody will agree that unit-tests > >>>> must > >>>>> be quite simple and very fast. Unit tests which takes more than > 3-5 > >>>> seconds > >>>>> should be refactored and/or moved to 'integration' tests. > >>>>> Thanks to Tom Barron for several fixes like [1]. IMO, we it would > be > >>>>> good to have some hacking checks to prevent such issues in a > future. > >>>>> > >>>>> - Tests coverage. We don't check it in an automatic way on gates. > >>>>> Usually, we require to add some unit-tests during code review > >>>> process. Why > >>>>> can't we add coverage job to our CI and do not merge new patches, > >> with > >>>>> will decrease tests coverage rate? Maybe, such job could be voting > >> in > >>>> a > >>>>> future to not ignore it. For now, there is not simple way to check > >>>> coverage > >>>>> because 'tox -e cover' output is not useful [2]. > >>>>> > >>>>> > >>>>> Functional tests for Cinder > >>>>> > >>>>> We introduced some functional tests last month [3]. Here is a patch > to > >>>>> infra to add new job [4]. Because these tests were moved from > >>>> unit-tests, I > >>>>> think we're OK to make this job voting. Such tests should not be a > >>>>> replacement for Tempest. They even could tests Cinder with Fake > Driver > >> to > >>>>> make it faster and not related on storage backends issues. > >>>>> > >>>>> > >>>>> Tempest in-tree tests > >>>>> > >>>>> Sean started work on it [5] and I think it's a good idea to get them > in > >>>>> Cinder repo to run them on Tempest jobs and 3-rd party CIs against a > >> real > >>>>> backend. > >>>>> > >>>>> > >>>>> Functional tests for python-brick-cinderclient-ext > >>>>> > >>>>> There are patches that introduces functional tests [6] and new job > [7]. > >>>>> > >>>>> > >>>>> Functional tests for python-cinderclient > >>>>> > >>>>> We've got a very limited set of such tests and non-voting job. IMO, > we > >>>> can > >>>>> run them even with Cinder Fake Driver to make them not depended on a > >>>>> storage backend and make it faster. I believe, we can make this job > >>>> voting > >>>>> soon. Also, we need more contributors to this kind of tests. > >>>>> > >>>>> > >>>>> Integrated tests for python-cinderclient > >>>>> > >>>>> We need such tests to make sure that we won't break Nova, Heat or > other > >>>>> python-cinderclient consumers with a next merged patch. There is a > >> thread > >>>>> in openstack-dev ML about such tests [8] and proposal [9] to > introduce > >>>> them > >>>>> to python-cinderclient. > >>>>> > >>>>> > >>>>> Rally tests > >>>>> > >>>>> IMO, it would be good to have new Rally scenarios for every patches > >> like > >>>>> 'improves performance', 'fixes concurrency issues', etc. Even if we > as > >> a > >>>>> Cinder community don't have enough time to implement them, we have to > >> ask > >>>>> for them in reviews, openstack-dev ML, file Rally bugs and blueprints > >> if > >>>>> needed. > >>>>> > >>>> > >>>> Are there any recent examples of a fix like this recently where it > would > >>>> seem like a reasonable task to write a Rally scenario along with the > >> patch? > >>>> > >>>> Not being very familiar with Rally (as I think most of us aren't), I'm > >>>> having a hard time picturing this. > >>>> > >>>>> > >>>>> [1] https://review.openstack.org/#/c/282861/ > >>>>> [2] http://paste.openstack.org/show/488925/ > >>>>> [3] https://review.openstack.org/#/c/267801/ > >>>>> [4] https://review.openstack.org/#/c/287115/ > >>>>> [5] https://review.openstack.org/#/c/274471/ > >>>>> [6] https://review.openstack.org/#/c/265811/ > >>>>> [7] https://review.openstack.org/#/c/265925/ > >>>>> [8] > >>>>> > >>>> > >> > http://lists.openstack.org/pipermail/openstack-dev/2016-March/088027.html > >>>>> [9] https://review.openstack.org/#/c/279432/ > >>>>> > >>>>> > >>>>> Regards, > >>>>> Ivan Kolodyazhny, > >>>>> http://blog.e0ne.info/ > >>>>> > >> > >> > >> > __________________________________________________________________________ > >> OpenStack Development Mailing List (not for usage questions) > >> Unsubscribe: > [email protected]?subject:unsubscribe > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > > > > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > [email protected]?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
