My tempest experience is dated to Havana, so I'll consider it to be completely out of date. Glad to he about all these improvements. On Jul 13, 2015 2:12 PM, "Matthew Treinish" <[email protected]> wrote:
> On Sun, Jul 12, 2015 at 09:29:57PM -0600, Matt Fischer wrote: > > We used Tempest for a time against our production environment. It was a > > pain to clean up but ephemeral test jobs solves that for you. A few > > questions > > Just curious how long ago was this? Because resource leaks have always been > treated as bugs in tempest. We've done a lot of work over the past year to > get > better about our cleanup in tempest. There are also config options to > limit the > dynamic creation of resources and provide a static set of user/tenants to > limit any potential dangling users. [1] Although we should always be > calling > delete for created users and projects as part of test setup. (this had to > be > fixed for other internal mechanisms to work) > > I'm just looking to get more feedback about this, because I see statements > like > this from time to time but not many bugs being filed about specific issues > people encounter. > > > > > What version of tempest will be using? > > Will we maintain a blacklist if there are known failures? (although this > is > > a pain to keep updated) > > Do you mean because of configuration differences you don't want to run > certain > tests or because of bugs in OpenStack causing intermittent failures? > > -Matt Treinish > > [1] > http://docs.openstack.org/developer/tempest/configuration.html#locking-test-accounts-aka-accounts-yaml-or-accounts-file > > > > > On Sun, Jul 12, 2015 at 8:44 PM, Emilien Macchi < > [email protected]> > > wrote: > > > > > Hi, > > > > > > I would like to propose to run Tempest at the end of the beaker jobs, > for > > > testing purpose now. > > > > > > As a start, we would accept 0 & 1 as return code, because this is > really > > > experimental. > > > Though I think it will be interesting to see how it behaves, specially > > > when we implement new configurations or plugins in our modules. > > > > > > I already did a PoC for puppet-keystone: > > > https://review.openstack.org/198561 > > > (failing because it needs more work to pass keystone v3 tests but v2 > tests > > > are successful). > > > As you can see the code is really light, since we use puppet-tempest. > > > > > > Any suggestion is welcome ! > > > > > > -- > > > Emilien Macchi > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
