Re: [openstack-dev] [tripleo][ui][tempest][oooq] Refreshing plugins from git
Here is an etherpad with all of the open patches that Chandan and I have been working on. https://etherpad.openstack.org/p/selenium-testing-ci On 2018-10-22 18:25, Chandan kumar wrote: > Hello Honza, > > On Thu, Oct 18, 2018 at 6:15 PM Bogdan Dobrelya wrote: > > > > On 10/18/18 2:17 AM, Honza Pokorny wrote: > > > Hello folks, > > > > > > I'm working on the automated ui testing blueprint[1], and I think we > > > need to change the way we ship our tempest tests. > > > > > > Here is where things stand at the moment: > > > > > > * We have a kolla image for tempest > > > * This image contains the tempest rpm, and the openstack-tempest-all rpm > > > * The openstack-tempest-all package in turn contains all of the > > >openstack tempest plugins > > > * Each of the plugins is shipped as an rpm > > > > > > So, in order for a new test in tempest-tripleo-ui to appear in CI we > > > have to go through at least the following tests: > > > > > > * New tempest-tripleo-ui rpm > > > * New openstack-tempest-all rpm > > > * New tempest kolla image > > > > > > This could easily take a week, if not more. > > > > > > What I would like to build is something like the following: > > > > > > * Add an option to the tempest-setup.sh script in tripleo-quickstart to > > >refresh all tempest plugins from git before running any tests > > > * Optionally specify a zuul change for any of the plugins being > > >refreshed > > > * Hook up the test job to patches in tripleo-ui (which tests in > > >tempest-tripleo-ui are testing) so that I can run a fix and its test > > >in a single CI job > > I have added a patch in TripleO Quickstart extras Validate-tempest > role: https://review.openstack.org/#/c/612377/ to install any tempest > plugin from git and zuul will pick > the specific change in the gates. > Here is the patch on how to test it with FS: > https://review.openstack.org/612386 > Basically in any FS, we can add following lines > tempest_format: venv > tempest_plugins_git: >- 'https://git.openstack.org/openstack/tempest-tripleo-ui.git' > the respective FS related job will install the tempest plugin and we > can also use test_white_regex: to > trigger the tempest tests. > > I think it will solve the problem. > > Thanks > > Chandan Kumar > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo][ui][tempest][oooq] Refreshing plugins from git
Hello Honza, On Thu, Oct 18, 2018 at 6:15 PM Bogdan Dobrelya wrote: > > On 10/18/18 2:17 AM, Honza Pokorny wrote: > > Hello folks, > > > > I'm working on the automated ui testing blueprint[1], and I think we > > need to change the way we ship our tempest tests. > > > > Here is where things stand at the moment: > > > > * We have a kolla image for tempest > > * This image contains the tempest rpm, and the openstack-tempest-all rpm > > * The openstack-tempest-all package in turn contains all of the > >openstack tempest plugins > > * Each of the plugins is shipped as an rpm > > > > So, in order for a new test in tempest-tripleo-ui to appear in CI we > > have to go through at least the following tests: > > > > * New tempest-tripleo-ui rpm > > * New openstack-tempest-all rpm > > * New tempest kolla image > > > > This could easily take a week, if not more. > > > > What I would like to build is something like the following: > > > > * Add an option to the tempest-setup.sh script in tripleo-quickstart to > >refresh all tempest plugins from git before running any tests > > * Optionally specify a zuul change for any of the plugins being > >refreshed > > * Hook up the test job to patches in tripleo-ui (which tests in > >tempest-tripleo-ui are testing) so that I can run a fix and its test > >in a single CI job I have added a patch in TripleO Quickstart extras Validate-tempest role: https://review.openstack.org/#/c/612377/ to install any tempest plugin from git and zuul will pick the specific change in the gates. Here is the patch on how to test it with FS: https://review.openstack.org/612386 Basically in any FS, we can add following lines tempest_format: venv tempest_plugins_git: - 'https://git.openstack.org/openstack/tempest-tripleo-ui.git' the respective FS related job will install the tempest plugin and we can also use test_white_regex: to trigger the tempest tests. I think it will solve the problem. Thanks Chandan Kumar __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo][ui][tempest][oooq] Refreshing plugins from git
On 10/18/18 2:17 AM, Honza Pokorny wrote: Hello folks, I'm working on the automated ui testing blueprint[1], and I think we need to change the way we ship our tempest tests. Here is where things stand at the moment: * We have a kolla image for tempest * This image contains the tempest rpm, and the openstack-tempest-all rpm * The openstack-tempest-all package in turn contains all of the openstack tempest plugins * Each of the plugins is shipped as an rpm So, in order for a new test in tempest-tripleo-ui to appear in CI we have to go through at least the following tests: * New tempest-tripleo-ui rpm * New openstack-tempest-all rpm * New tempest kolla image This could easily take a week, if not more. What I would like to build is something like the following: * Add an option to the tempest-setup.sh script in tripleo-quickstart to refresh all tempest plugins from git before running any tests * Optionally specify a zuul change for any of the plugins being refreshed * Hook up the test job to patches in tripleo-ui (which tests in tempest-tripleo-ui are testing) so that I can run a fix and its test in a single CI job This would allow the tripleo-ui team to develop code and tests at the same time, and prevent breakage before a patch is even merged. Here are a few questions: * Do you think this is a good idea? This reminds the update_containers case, but relaxed the next level of updating from sources instead of rpm. Given that we already have that update_containers thing, the idea seems acceptable for CI use only. Although I'd prefer to see the packages and the tempest container (and all that update_containers affects) rebuilt in the same CI job run instead. Though I'm not sure for having different paths for "new test in tempest-tripleo-ui" getting into container: executed in CI vs executed via TripleO UI? I think the path it takes should always be the same. But please excuse me if I got the case wrong. [0] https://goo.gl/5bCWRX * Could we accomplish this by some other, simple mechanism? Any helpful suggestions, corrections, and feedback are much appreciated. Thanks Honza Pokorny [1]: https://blueprints.launchpad.net/tripleo/+spec/automated-ui-testing __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best regards, Bogdan Dobrelya, Irc #bogdando __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
