On 03/11/16 13:04, Greg Sutcliffe wrote: >> ---------- Forwarded message ---------- >> From: Stephen Benjamin <[email protected] <mailto:[email protected]>> >> Date: 3 October 2016 at 21:28 >> Subject: [foreman-dev] Plugin test failures >> To: Foreman <[email protected] > <mailto:[email protected]>> >> >> Would it be possible to get a standard plugin in Foreman's test matrix > for PR? Perhaps tasks is a good candidate. Not on all databases/rubies, > but something might help gain awareness of the impact of certain changes > on plugins. Katello is there, but it doesn't run all (any?) of Foreman's > tests. > > I think this is a good plan. I'd be happy to see tasks or maybe salt > added to the test matrix in the short term. I also think it might make > sense to have a dedicated "fake" plugin that touches a whole ton of > stuff just for the sake of knowing if we've broken it. We could look to > that for the longer term.
Foreman already has a test suite for its plugin interfaces which is run on every pull request. If those tests are insufficient, expand them. > I might be wrong, but I don't think we merge breaking changes after > freeze-date, so it should be possible to track this and then tell > authors about the changes when the first RCs go out Deprecations affecting plugins are listed in release notes today, though some plugin authors don't seem to bother changing things until the interfaces are later removed. > 1) Test resources - do we have enough to run this on every PR? Do we > want to, or should it be less frequent? There's no need. The Foreman plugin interface is already tested on every PR, and plugin tests themselves are run at least weekly - authors should be aware of issues quickly. You should file a bug against Foreman immediately if you see that a plugin interface has been broken, so we can ensure the change isn't released and a fix can be written. -- Dominic Cleal [email protected] -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
