On Thu, Jun 1, 2017 at 2:10 PM, Marek Hulán <[email protected]> wrote:
> On čtvrtek 1. června 2017 12:17:25 CEST Lukas Zapletal wrote: > > Hello, > > > > I would like to open discussion about having a unit test job of > > foreman with all major plugins enabled (katello, discovery, bootdisk, > > rex, openscap). Some of the bugs we found in Discovery 9.0 would have > > been found just by executing unit tests with all the plugins enabled. > > > > If this is a problem of compute time, let's simply schedule a daily > > job, so at least we know if nightly is stable or not with all the > > plugins enabled. > > I agree this became a must. Since 1.15 compose, foreman_templates relies on > remote_execution template import method which was changed in last version. > foreman_openscap seeds remote execution job templates, again affected by > recent change. Result was installer fails with openscap activated, because > seed failed. Luckily we found it our dev setups before the release but we > should know whether plugins work together. Otherwise after new stable core > version is released, we're fixing all plugin for another month. > > Based on community survey [1], 89% of our users use plugins. We should > treat > at least most popular ones as part of the core. We already have a way for > plugins to disable tests, that can't run when the plugin is active. I know > that e.g. Katello breaks ~200 of Foreman tests so we'd need to fix this > first, > but I definitely support this and I'm happy to help with it. > > +1 I suggest that's the hard investment we would need to do upfront, e.g. fix all plugins and not just blindly disable tests. > [1] https://theforeman.org/2017/03/2017-foreman-survey-analysis.html#page2 > > -- > Marek > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
