On 07/05/2017 10:21 AM, Emilien Macchi wrote:
On Wed, Jul 5, 2017 at 7:57 AM, Florian Fuchs <flfu...@redhat.com> wrote:
On Tue, Jul 4, 2017 at 7:48 PM, Emilien Macchi <emil...@redhat.com> wrote:
On Fri, Jun 30, 2017 at 10:51 AM, Honza Pokorny <ho...@redhat.com> wrote:
I'd like to write some integration tests for the TripleO UI.  Simple
configuration checks, workflow tests, websocket tests, etc.

I had a quick look at the tripleo-ci [1] repository, and it wasn't
immediately obvious to me how to extend the existing infrastructure to
add something like this.  It seems to me that the current way is just
"stick some shell script here and here".  The tripleo.sh script is about
1,600 LOC and it look rather intimidating.

I'm looking for help or pointers on how to do this.  Is there a standard
way of accomplishing this?  Are there any helpers I should be aware of?
Is there any documentation beyond what is in the git tree?

Any help would be appreciated

I think we have 2 options here (which could also be mixed):

- Using tripleo-validatons and make sure we run it in CI.

+1 for using ansible playbooks to write UI tests. I just don't think
we should host the test playbooks in the tripleo-validations
repository, because IMO testing the UI is different to what the
tripleo-validations are doing.
That said, the UI tests could use the ansible inventory from
tripleo-validations -- which ends up in
/usr/bin/tripleo-ansible-inventory and is already being used by
playbooks other than tripleo-validations.

If tripleo-ui functional tests are written in Python, it makes a lot
of sense to write a Tempest plugin instead of using Ansible.

+1. This seems like an obvious place to take advantage of the framework Tempest already has in place. These aren't intended as user-facing tests so implementing them as validations would be a weird fit to me.

The major reason to me is consistency with how we test other services
in TripleO.

Whatever option we'll take, we'll need a new repository I think that
will contain the actual tests. There is a goal in Queens where Tempest
plugins are out of tree:
https://governance.openstack.org/tc/goals/queens/split-tempest-plugins.html
If we take that path, we'll create a new repo for that.


The Ansible option would also be fine but we need to acknowledge the
fact tripleo-ui testing wouldn't be possible without deploying the
tripleo-validations bits.

Florian


- Write a tempest plugin for tripleo-ui (like Horizon has one:
https://github.com/openstack/tempest-horizon) (it could be
openstack/tempest-triploe-ui) for example. We could run the tempest
tests in the CI jobs (we're working on running Tempest on scenario
jobs, see efforts made in
https://trello.com/c/Z8jIillp/252-spec-out-the-work-required-to-run-tempest-on-the-tripleo-gating-jobs).

Please do not implement tests in tripleo-ci or quickstart, I think we
need a way to test it outside CI and include it directly in the
product.

Thanks,

Thanks!

Honza Pokorny

[1]: https://github.com/openstack-infra/tripleo-ci

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Emilien Macchi

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to