I don't really have much context in what the decision is going to be based on here, so I'll just add some random comments here and there.
On Thu, Apr 6, 2017 at 12:48 PM Arx Cruz <arxc...@redhat.com> wrote: > Having tempest running will allow these jobs to appear in openstack-health > system as well. > I agree that's a plus. It's also rather easy to produce subunit from whatever you are using to run tests, and that's all you need in fact to get data into open stack-health without touching the existing infrastructure. So in case you decide not to use Tempest, open stack-health can still be on the list. > > On Thu, Apr 6, 2017 at 1:29 PM, Justin Kilpatrick <jkilp...@redhat.com> > wrote: > > Maybe I'm getting a little off topic with this question, but why was > Tempest removed last time? > > I'm not well versed in the history of this discussion, but from what I > understand Tempest in the gate has > been an off and on again thing for a while but I've never heard the > story of why it got removed. > > On Thu, Apr 6, 2017 at 7:00 AM, Chris Dent <cdent...@anticdent.org> wrote: > > On Thu, 6 Apr 2017, Sagi Shnaidman wrote: > > > >> It may be useful to run a "limited edition" of API tests that maximize > >> coverage and don't duplicate, for example just to check service working > >> basically, without covering all its functionality. It will take very > >> little > >> time (i.e. 5 tests for each service) and will give a general picture of > >> deployment success. It will cover fields that are not covered by > pingtest > >> as well. > > > > > We have a smoke attribute here an there, but it's not well curated at all, so you're probably better off maintaining your own list. Since presumably you're more interested in verifying that a deployed cloud is functional - as opposed to verify specific APIs are working properly - you may want to look at scenario tests, where with a couple of test you can cover already a lot of basic stuff, e.g. if you can boot a server from a volume with an image from glance, and ssh into it, you have proven a lot already about the general health of your cloud. > > > > It's sound like using some parts of tempest is perhaps the desired > > thing here but in case a "limited edition" test against the APIs to > > do what amounts to a smoke test is desired, it might be worthwhile > > to investigate using gabbi[1] and its command line gabbi-run[2] tool for > > some fairly simple and readable tests that can describe a sequence > > of API interactions. There are lots of tools that can do the same > > thing, so gabbi may not be the right choice but it's there as an > > option. > > > > The telemetry group had (an may still have) some integration tests > > that use gabbi files to integrate ceilometer, heat (starting some > > vms), aodh and gnocchi and confirm that the expected flow happened. > > Since the earlier raw scripts I think there's been some integration > > with tempest, but gabbi files are still used[3]. > > > > If this might be useful and I can help out, please ask. > > > > [1] http://gabbi.readthedocs.io/ > > [2] http://gabbi.readthedocs.io/en/latest/runner.html > > [3] > > > https://github.com/openstack/ceilometer/tree/master/ceilometer/tests/integration > > > > -- > > Chris Dent ¯\_(ツ)_/¯ https://anticdent.org/ > > freenode: cdent tw: @anticdent > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev