Re: [foreman-dev] Re: Unit test jenkins job with all plugins enabled

[Eric D Helms]

On Thu, Jun 1, 2017 at 10:18 AM, Tomer Brisker  wrote:

> Hello,
>
> While it is plugin maintainers responsibility to keep their plugins up to
> date, one of the most common frustrations they face is "core changed and
> broke something AGAIN". I wouldn't want to slow down core's already slow
> merge process, but at the same time core contributors should try to
> minimize breaking plugins as much as possible - for example, by deprecating
> functions before removing them completely. We could make these tests
> optional - i.e. not block merge if plugin tests fail, but just take
> advantage of the tests to fix whatever is needed in the plugins sooner
> rather then later. I know that in certain cases having katello test run on
> every PR let us find and fix issues we wouldn't have known about otherwise.
> Right now contributors (and reviewers) don't have any visibility to
> whether a PR will cause problems to other plugins. If breaking is
> unavoidable, at least having plugin tests run on PRs will allow giving a
> "heads up" to plugin maintainers that they need to fix something, rather
> then realizing after the next release that something broke.
> During the past couple of months we had several cases of core PRs leading
> to broken plugins, which could have been prevented or at least handled
> better had we known that a certain change requires changes in plugins.
>

Agree! If I had to sum it up, developer happiness and innovation at speed
(core and plugins) is key! One option we've mentioned before that goes
directly to Tomer's point is a stable plugin API. We as a community both
develop and tell other developers, come, build a plugin. We have a vibrant
plugin community. We say plugins are central to the community and project
yet CI/CD tells a different story. Do we have a defined contract for
plugins? That would solve a lot of the issues IMO. If we had a supported
interface (for ALL major use cases of extension, including the UI, since we
do have a Foreman::Plugin today) then core could make changes at will as
long as the interface passed testing. Any plugin using functionality not in
the interface would be SOL but it would be explicitly stated and known.

Once we get to installation, running in production mode, scenario testing
things get dicier, less stable and with less testing. A good place to
invest and bring in other communities (hint: Satlelite QE) to bolster. I'm
writing a lot of words without action items which I typically try to avoid.
I want to get our testing and matrices there and am working to bring about
more time for myself and others to do so. We need to actively invest in
this area, make it a priority, and a first class citizen of our community
to make an impact.

We have wiki pages on creating and releasing plugins. But maybe we need to
step back and define what a plugin is to our community. I think it would
also be a fruitful effort to examine the landscape of plugins and see how
and where they are creating "interfaces" to Foreman.


Eric


>
> As for which plugins to test - I think the best criteria would be to test
> the 5 or 10 most downloaded plugins or most used plugins according to the
> community survey.
>
> On Thu, Jun 1, 2017 at 4:50 PM, Lukas Zapletal  wrote:
>
>> On Thu, Jun 1, 2017 at 2:55 PM, Eric D Helms 
>> wrote:
>> > This sounds good in theory, but I'll refresh the history of why we did
>> this.
>> > A change would be made to Foreman core, that change would go in and then
>> > propagate out. A developer would update their foreman checkout or the
>> > Katello pipeline would run and break. More developers start updating
>> local
>> > checkouts and now everyone is broken. The Katello PR tests begin to
>> fail and
>> > all PRs are frozen. So now a significant portion of developers are all
>> > trying to figure out what point in time to roll back to, who is going
>> to fix
>> > the breakage and how long it is going to take. 1-5 days later, the
>> issue is
>> > fixed and PRs are able to start back up as well as the nightly pipeline.
>> >
>> > How much developer frustration was created? How much time was lost and
>> > wasted? If we want core to have more freedom of movement then core
>> needs to
>> > evolve to allow plugins more reliability on interfacing.
>>
>> Then let's don't block PRs, I propose nightly job with an email so we
>> are at least aware that things do not work and we can at least wait
>> with RC phase a little bit longer until things settle down, or even
>> block doing another RC until we are all green.
>
>
You lost me here. Blocking PRs is the only way that we reduced developer
frustration. Not blocking means broken code, broken installs, broken
developer environments. Say core merges something that removes a function
Katello relied on. But we did not run Katello unit tests or block at all.
Now a significant chunk of developers can't even boot their application.
Meanwhile, we didn't block PRs due to 

Re: [foreman-dev] Katello 3.4 sync time

[Eric D Helms]

On Thu, Jun 1, 2017 at 10:01 AM, Lukas Zapletal  wrote:

> > Why is this sub-par performance? How long do you think it should take? Is
> > this an increase over previous? Initial processing time, whether
> on-demand
> > or not, is typically the longest sync you'll encounter for a repository
> > given all of the data that has to be downloaded and processed.
>
> We are taking metadata from CDN/upstream repo and simply "copying"
> them into Pulp, there is no filtering applied or anything. It looks
> like Pulp processes this "from scratch" while it could simply "trust"
> the upstream metadata and just copy it? Then I'd expect minutes, but
> not forty.
>

This is getting into more a discussion that should be had with the Pulp
project. But the from scratch is also to pull information into their
database to make it available for querying, viewing whats in a repository
and being able to process it for the streamer service so it appears like a
normal repository but without the bits being on disk until requested.

Eric


>
> I understand there is some design in Pulp and some processing is going
> on there for the initial sync, but I expected on-demand policy to be
> even faster. I wonder if there are any plans improving initial sync in
> Pulp3 maybe.
>
> BK: I haven't compared with mrepo/createrepo or createrepo_c.
>
> --
> Later,
>   Lukas @lzap Zapletal
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Eric D. Helms
Red Hat Engineering

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Dropping Ruby 2.0 support in 1.16?

[Tomer Brisker]

Perhaps we could consider giving certain versions longer support cycles
then usual?
Not 2 years, but maybe instead of the usual 6 months make it 9 so that
users stuck because they can't upgrade to e.g. stretch yet won't be stuck
with a completely unmaintained version so quickly? Obviously only
significant bugfixes will be backported during this period, but it would
give our users a bit more time to upgrade their infra.

On Thu, Jun 1, 2017 at 3:30 PM, Michael Moll  wrote:

> Hi,
>
> On Thu, Jun 01, 2017 at 02:49:12PM +0300, Ohad Levy wrote:
> > My main concern here is that we would need to support 4.2 and 5 for a
> long
> > period of time without using any of the benefits in 5 (more work - less
> > features), what can be even worse, is that some of the bugs in rails will
> > be fixed only in 5 and then we would have some inconsistency across
> > distributions.
>
> I didn't say it's going to be fun... ;)
>
> > How much longer do you think we should announce in advance before
> > announcing that a given release is the last supported one for a given
> > distribution?
>
> Ideally I'd like to see Ruby 2.1 supported until 1.17. While requiring
> newer nodejs versions isn't a problem thanks to the nodesource
> repositories, there's nothing similar for newer Ruby versions.
>
> However, if Rails 5 is really high priority, yes, then there's no way
> around getting rid of Ruby 2.1 after the 1.16 release and we should
> announce that as early as possible. I'm just seeing quite some users
> stuck with 1.16 on Debian 8 then - like some are still on 1.12/1.13
> because of EL6.
>
> Regards
> --
> Michael Moll
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Have a nice day,
Tomer Brisker
Red Hat Engineering

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Unit test jenkins job with all plugins enabled

[Marek Hulán]

On čtvrtek 1. června 2017 12:17:25 CEST Lukas Zapletal wrote:
> Hello,
> 
> I would like to open discussion about having a unit test job of
> foreman with all major plugins enabled (katello, discovery, bootdisk,
> rex, openscap). Some of the bugs we found in Discovery 9.0 would have
> been found just by executing unit tests with all the plugins enabled.
> 
> If this is a problem of compute time, let's simply schedule a daily
> job, so at least we know if nightly is stable or not with all the
> plugins enabled.

I agree this became a must. Since 1.15 compose, foreman_templates relies on 
remote_execution template import method which was changed in last version. 
foreman_openscap seeds remote execution job templates, again affected by 
recent change. Result was installer fails with openscap activated, because 
seed failed. Luckily we found it our dev setups before the release but we 
should know whether plugins work together. Otherwise after new stable core 
version is released, we're fixing all plugin for another month.

Based on community survey [1], 89% of our users use plugins. We should treat 
at least most popular ones as part of the core. We already have a way for 
plugins to disable tests, that can't run when the plugin is active. I know 
that e.g. Katello breaks ~200 of Foreman tests so we'd need to fix this first, 
but I definitely support this and I'm happy to help with it.

[1] https://theforeman.org/2017/03/2017-foreman-survey-analysis.html#page2

--
Marek

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Dropping Ruby 2.0 support in 1.16?

[Ohad Levy]

On Thu, Jun 1, 2017 at 1:53 PM, Tomer Brisker  wrote:

> Considering no objections were raised, as a first step I have opened a PR
> to disable Ruby 2.0 tests for core & plugins:
> https://github.com/theforeman/foreman-infra/pull/296
>
> I believe that 1.16 should require 2.1 at least, and I hope that we could
> figure out a way to get 1.17 to only support ruby 2.2.2 or newer so that we
> can upgrade to rails 5.
>

Speaking of rails 5, you should be aware that Dominic suggested a PR [1]
that enables both rails 4.2 and 5.0 at the same time based on the ruby
version used, this will allow us to run both 4 and 5 at the same time until
we drop support for older ruby versions.

[1] https://github.com/theforeman/foreman/pull/4422

>
> On Thu, May 18, 2017 at 6:15 PM, Ewoud Kohl van Wijngaarden <
> ew...@kohlvanwijngaarden.nl> wrote:
>
>> On Thu, May 18, 2017 at 10:57:03AM -0400, Stephen Benjamin wrote:
>> > On Wed, May 17, 2017 at 10:45 AM, Michael Moll 
>> wrote:
>> > > Hi,
>> > >
>> > > On Wed, May 17, 2017 at 03:11:12PM +0300, Tomer Brisker wrote:
>> > >> What do people think about dropping it in 1.16? This will still give
>> people
>> > >> enough time to upgrade their systems as 1.15 will still be supported
>> for
>> > >> the next 6 months.
>> > >
>> > > +1 on dropping 2.0 support for Foreman core.
>> > > --
>> >
>> > For the installer, we're going to be in the same painful situation as
>> > we were with 1.8.7. EL7 still has quite a few years left, what are the
>> > chances to get it inside SCL or (urgh) fully into Puppet AIO ruby
>> > environment?
>>
>> Given we no longer directly interface with puppet code I don't think we
>> need to move into the Puppet AIO env. I'd also like to avoid that. I'm
>> also not sure how many changes we'll have to make to keep it running on
>> 2.0 but others will have more insight into that.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Have a nice day,
> Tomer Brisker
> Red Hat Engineering
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] Unit test jenkins job with all plugins enabled

[Lukas Zapletal]

Hello,

I would like to open discussion about having a unit test job of
foreman with all major plugins enabled (katello, discovery, bootdisk,
rex, openscap). Some of the bugs we found in Discovery 9.0 would have
been found just by executing unit tests with all the plugins enabled.

If this is a problem of compute time, let's simply schedule a daily
job, so at least we know if nightly is stable or not with all the
plugins enabled.

-- 
Later,
  Lukas @lzap Zapletal

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] Re: [Event] Foreman Community Demo Items - Thu 01 Jun 3pm [GMT]

[Greg Sutcliffe]

Hi all,

Sadly I'm going to have to cancel today's demo - we have very few
presenters, so it would be very short anyway. Hopefully we'll have more
content for you next time!

Bernhard, I'll contact you separately about a Deep Dive for the SuSE
plugin. I think it warrants some attention :)

Greg

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.