This came up on 3rd Oct, but I think the second half of Stephen's email got lost in the discussion of the immediate issues. To summarize:
> ---------- Forwarded message ---------- > From: Stephen Benjamin <[email protected]> > Date: 3 October 2016 at 21:28 > Subject: [foreman-dev] Plugin test failures > To: Foreman <[email protected]> > > Would it be possible to get a standard plugin in Foreman's test matrix for PR? Perhaps tasks is a good candidate. Not on all databases/rubies, but something might help gain awareness of the impact of certain changes on plugins. Katello is there, but it doesn't run all (any?) of Foreman's tests. I think this is a good plan. I'd be happy to see tasks or maybe salt added to the test matrix in the short term. I also think it might make sense to have a dedicated "fake" plugin that touches a whole ton of stuff just for the sake of knowing if we've broken it. We could look to that for the longer term. > Along side that, we should be tracking what plugin maintainers need to do to update from one release to the next. This is good too - I'll be honest and say that my plugins don't seem to break as often as yours (just luck I guess) but it is useful when people come tell me what needs updating. I might be wrong, but I don't think we merge breaking changes after freeze-date, so it should be possible to track this and then tell authors about the changes when the first RCs go out - but first we need the "fake" plugin that touches tons of stuff, so we know what the diffs look like (we could even just send a list of commits to the fake plugin, showing how to change the methods). I can see the main concerns being: 1) Test resources - do we have enough to run this on every PR? Do we want to, or should it be less frequent? 2) Should it block merges, or be advisory only? If yes, do PR authors need to prepare patches for the plugin when they break a plugin? I'm happy to write this up as an RFC if others think its worth having a more formal discussion, or if one of the people with access wants to give running foreman_tasks/salt on PRs a trial, that's cool too. Greg -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
