On Mon, Apr 17, 2017 at 7:58 PM Ihar Hrachyshka <ihrac...@redhat.com> wrote:
> On Mon, Apr 17, 2017 at 9:35 AM, Jordan Pittier > <jordan.pitt...@scality.com> wrote: > > We don"t run slow tests because the QA team think that they don't bring > > enough value to be executed, every time and everywhere. The idea is that > if > > some specific slow tests are of some interest to some specific openstack > > projects, those projects can change the config of their jobs to enable > these > > tests. > > > But since it's not executed anywhere in tempest gate, even as > non-voting (?), it's effectively dead code that may be long broken > without anyone knowing. Of course there are consumers of the tests > downstream, but for those consumers it's a tough call to start > depending on the tests if they are not sanity checked by tempest > itself. Wouldn't it make sense to have some job in tempest gate that > would execute those tests (maybe just them to speed up such a job? > maybe non-voting? We reduced the number of scenario tests we run in the main gate because some of those tests are not so critical for the integrated gate, and also because we had an unstable gate we had to deal with to allow for Pike development to happen. The SUT was overloaded in the gate, due to a combination of effects: too many services running, many consuming more memory that it did in the past, as well as too many "heavy" tests running in parallel. As soon as removed scenario tests from the main gate we introduced a new non-voting job against Tempest, which runs *all* scenario tests against a multimode environment, see for instance [0], so we can ensure the code is not dead as long as it lives in Tempest. I'm planning to add more "heavy/long" tests to that jobs, e.g. compute migrate ones [1]. andreaf [0] http://logs.openstack.org/69/451769/2/check/gate-tempest-dsvm-neutron-scenario-multinode-ubuntu-xenial-nv/1f71416/ [1] https://review.openstack.org/#/c/451769/ > maybe even as periodic? but there should be > something that keeps it green in long run). > > Ihar > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev