On Thu, Jul 6, 2017 at 6:07 AM, Honza Pokorny <[email protected]> wrote: > On 2017-07-05 12:42, Ben Nemec wrote: >> >> >> On 07/05/2017 10:21 AM, Emilien Macchi wrote: >> > On Wed, Jul 5, 2017 at 7:57 AM, Florian Fuchs <[email protected]> wrote: >> > > On Tue, Jul 4, 2017 at 7:48 PM, Emilien Macchi <[email protected]> >> > > wrote: >> > > > On Fri, Jun 30, 2017 at 10:51 AM, Honza Pokorny <[email protected]> >> > > > wrote: >> > > > > I'd like to write some integration tests for the TripleO UI. Simple >> > > > > configuration checks, workflow tests, websocket tests, etc. >> > > > > >> > > > > I had a quick look at the tripleo-ci [1] repository, and it wasn't >> > > > > immediately obvious to me how to extend the existing infrastructure >> > > > > to >> > > > > add something like this. It seems to me that the current way is just >> > > > > "stick some shell script here and here". The tripleo.sh script is >> > > > > about >> > > > > 1,600 LOC and it look rather intimidating. >> > > > > >> > > > > I'm looking for help or pointers on how to do this. Is there a >> > > > > standard >> > > > > way of accomplishing this? Are there any helpers I should be aware >> > > > > of? >> > > > > Is there any documentation beyond what is in the git tree? >> > > > > >> > > > > Any help would be appreciated >> > > > >> > > > I think we have 2 options here (which could also be mixed): >> > > > >> > > > - Using tripleo-validatons and make sure we run it in CI. >> > > >> > > +1 for using ansible playbooks to write UI tests. I just don't think >> > > we should host the test playbooks in the tripleo-validations >> > > repository, because IMO testing the UI is different to what the >> > > tripleo-validations are doing. >> > > That said, the UI tests could use the ansible inventory from >> > > tripleo-validations -- which ends up in >> > > /usr/bin/tripleo-ansible-inventory and is already being used by >> > > playbooks other than tripleo-validations. >> > >> > If tripleo-ui functional tests are written in Python, it makes a lot >> > of sense to write a Tempest plugin instead of using Ansible. >> >> +1. This seems like an obvious place to take advantage of the framework >> Tempest already has in place. These aren't intended as user-facing tests so >> implementing them as validations would be a weird fit to me. > > OK, great, let's go with tempest. Initially, I was intrigued with the > idea of reusing some of the ansible code we already have in the > validations repository. But in the end, it seems like tempest is the > better tool for what we want.
Let's give it a try and see how Tempest works for us. I'm ok with baby steps and adapt if we realize it wasn't what we needed. > Emilien, how do we go about setting up the new project (GitHub, gerrit, > etc)? Long story: https://docs.openstack.org/infra/manual/creators.html Short story: https://review.openstack.org/#/c/459762/ / https://review.openstack.org/#/c/459993/ >> >> > The major reason to me is consistency with how we test other services >> > in TripleO. >> > >> > Whatever option we'll take, we'll need a new repository I think that >> > will contain the actual tests. There is a goal in Queens where Tempest >> > plugins are out of tree: >> > https://governance.openstack.org/tc/goals/queens/split-tempest-plugins.html >> > If we take that path, we'll create a new repo for that. >> > >> > >> > The Ansible option would also be fine but we need to acknowledge the >> > fact tripleo-ui testing wouldn't be possible without deploying the >> > tripleo-validations bits. >> > >> > > Florian >> > > >> > > >> > > > - Write a tempest plugin for tripleo-ui (like Horizon has one: >> > > > https://github.com/openstack/tempest-horizon) (it could be >> > > > openstack/tempest-triploe-ui) for example. We could run the tempest >> > > > tests in the CI jobs (we're working on running Tempest on scenario >> > > > jobs, see efforts made in >> > > > https://trello.com/c/Z8jIillp/252-spec-out-the-work-required-to-run-tempest-on-the-tripleo-gating-jobs). >> > > > >> > > > Please do not implement tests in tripleo-ci or quickstart, I think we >> > > > need a way to test it outside CI and include it directly in the >> > > > product. >> > > > >> > > > Thanks, >> > > > >> > > > > Thanks! >> > > > > >> > > > > Honza Pokorny >> > > > > >> > > > > [1]: https://github.com/openstack-infra/tripleo-ci >> > > > > >> > > > > __________________________________________________________________________ >> > > > > OpenStack Development Mailing List (not for usage questions) >> > > > > Unsubscribe: >> > > > > [email protected]?subject:unsubscribe >> > > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > >> > > > >> > > > >> > > > -- >> > > > Emilien Macchi >> > > > >> > > > __________________________________________________________________________ >> > > > OpenStack Development Mailing List (not for usage questions) >> > > > Unsubscribe: >> > > > [email protected]?subject:unsubscribe >> > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > >> > > __________________________________________________________________________ >> > > OpenStack Development Mailing List (not for usage questions) >> > > Unsubscribe: >> > > [email protected]?subject:unsubscribe >> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > >> > >> > >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: [email protected]?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Emilien Macchi __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
