Re: Opinions about aggressive reuse strategy for Jenkins pipelines

[David Karr]

I'm finding that simply moving the entire jenkinsfile into a single vars 
method works well enough, but they've taken this another step further by 
moving all the separate "def" methods called from the main method into 
separate vars methods.  I wouldn't think there'd be any technical reason 
why this would be a problem, but I'm occasionally seeing weird symptoms 
with these "second-level" vars methods. I find that sometimes it gets to 
the point where it calls one of those methods, and then just fails the 
build at that point without any diagnostic at all.  My workaround for those 
issues has been to simply make an exact copy of that second-level method 
and define it as a "def" method inside the main file, and that fixes the 
problem entirely. I didn't even change any of the code of the method, I 
just copied it into the main file, and that fixed the problem.

On Tuesday, April 9, 2019 at 9:37:32 AM UTC-7, David Karr wrote:
>
> I work for a very large organization, on a project with many teams, 
> working on many microservices, all of which were generated from a 
> home-grown application template system.  Most of the Java-based 
> microservices have a very similar build process. In fact, most of the 
> projects have an identical Jenkinsfile, with some small parameter value 
> differences (which are set in a different system, not in the Jenkinsfile).
>
> Over time, we've evolved the template and common features, including 
> modularizing and cleaning up the Jenkinsfile.  A small shared library was 
> developed, to hold some of the core function definitions, even though the 
> basic structure of the build was still "in-line" in the Jenkinsfile.
>
> The latest version of this template has taken reuse just about as far as 
> it can go.  The entire Jenkinsfile for most projects will be a small 
> properties block to override some defaults (that are not set in the other 
> system), along with a single method call, including the shared library 
> reference.  The shared library has now expanded to specify the entire build 
> process, from that one method call.
>
> There are a handful of services in the collection that will still have 
> some custom changes (all in my domain), so I will reference a branch of the 
> shared library in those Jenkinsfiles.  If my custom changes are eventually 
> accepted as standard (many of them have been), then they will be merged to 
> the master branch and I can change our Jenkinsfiles to use the master 
> branch.
>
> I'm describing all of this for background.  What I'm asking the community 
> is, is this a reasonable strategy?  I can't help the feeling that this is 
> taking reusability a bit too far. I can't give any technical arguments 
> against it, so I'm just looking for some perspective.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/fdcf2b2e-7c28-4183-93d5-0960c865ae0b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Opinions about aggressive reuse strategy for Jenkins pipelines

[Jan Monterrubio]

We’ve taken sort of a middle ground approach with this on our end. We
define standard flows for a particular build system (like maven)

The 80-90% use cases are allowable with our pipeline, consumers only define
eome configuration options in a closure.

For the extra 10% use case, we provide steps you can use “checkoutProject”
“buildSite” etc, so those non standard pipelines can still make use of our
functions.

This has helped us standardize our disjoint development process, on the
pipeline everyone deploys master but only runs verify on branches and PRs.
If that doesn’t fit your use case, and the configuration option doesn’t
work, we either:

1. Have a call for feedback on new option just to see how many of our
consumers need the new behavior
2. Help the consumer team use our functions to make their own pipeline.



On Wed, Apr 10, 2019 at 13:46 Ivan Fernandez Calvo 
wrote:

> It is a fair approach, however my opinion is that the pipeline definition
> (stages and steps) should be in the project pipeline (Jenkinsfile), in the
> shared library we have only steps that make only one thing an they make it
> well or steps that combine several others, every step has his own unit test
> and we release a new version of the library every Monday, all the
> jenkinsfiles load the ’current’ version of the library that it is the
> stable one. In that way we can go backwards between versions without change
> anything in projects. The reason to not put the whole pipeline in the
> library is that at some point you will start to write spaghetti pipeline
> code to cover the whole options needed to all your projects, when that
> fails would be a pain to debug such thing
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/e1ac1fd9-f494-4113-8483-f6f1c2bee739%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CADgiF9%2BowGKQoF8t4za2V_8goxTB1ARUbVCK3FbgisDKhKpwHQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Opinions about aggressive reuse strategy for Jenkins pipelines

[Ivan Fernandez Calvo]

It is a fair approach, however my opinion is that the pipeline definition 
(stages and steps) should be in the project pipeline (Jenkinsfile), in the 
shared library we have only steps that make only one thing an they make it well 
or steps that combine several others, every step has his own unit test and we 
release a new version of the library every Monday, all the jenkinsfiles load 
the ’current’ version of the library that it is the stable one. In that way we 
can go backwards between versions without change anything in projects. The 
reason to not put the whole pipeline in the library is that at some point you 
will start to write spaghetti pipeline code to cover the whole options needed 
to all your projects, when that fails would be a pain to debug such thing

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/e1ac1fd9-f494-4113-8483-f6f1c2bee739%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Opinions about aggressive reuse strategy for Jenkins pipelines

[Aaron Digulla]

Hi David

I'm describing all of this for background.  What I'm asking the community 
> is, is this a reasonable strategy?  I can't help the feeling that this is 
> taking reusability a bit too far. I can't give any technical arguments 
> against it, so I'm just looking for some perspective.
>

I think this is a sound strategy. Without this, projects will quickly 
become snowflakes. Every team will start to invent their own solution for 
similar problems. Many of those will be haphazard because people don't have 
time and/or knowledge to understand the underlying issue and correctly fix 
many problems. So the solutions tend to be brittle and there will be a lot 
of conversations: "Jenkins sucks. It always breaks" "What happened?" "Well 
... [long explanation]" "Hey, we had that too. We just [...]" "Oh."

Regards,

Aaron Digulla

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/9c086c8c-587b-43dd-9da0-5e2d6ee4319d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Opinions about aggressive reuse strategy for Jenkins pipelines

[Slide]

Sounds like a good system to me. :-)

On Tue, Apr 9, 2019 at 9:37 AM David Karr 
wrote:

> I work for a very large organization, on a project with many teams,
> working on many microservices, all of which were generated from a
> home-grown application template system.  Most of the Java-based
> microservices have a very similar build process. In fact, most of the
> projects have an identical Jenkinsfile, with some small parameter value
> differences (which are set in a different system, not in the Jenkinsfile).
>
> Over time, we've evolved the template and common features, including
> modularizing and cleaning up the Jenkinsfile.  A small shared library was
> developed, to hold some of the core function definitions, even though the
> basic structure of the build was still "in-line" in the Jenkinsfile.
>
> The latest version of this template has taken reuse just about as far as
> it can go.  The entire Jenkinsfile for most projects will be a small
> properties block to override some defaults (that are not set in the other
> system), along with a single method call, including the shared library
> reference.  The shared library has now expanded to specify the entire build
> process, from that one method call.
>
> There are a handful of services in the collection that will still have
> some custom changes (all in my domain), so I will reference a branch of the
> shared library in those Jenkinsfiles.  If my custom changes are eventually
> accepted as standard (many of them have been), then they will be merged to
> the master branch and I can change our Jenkinsfiles to use the master
> branch.
>
> I'm describing all of this for background.  What I'm asking the community
> is, is this a reasonable strategy?  I can't help the feeling that this is
> taking reusability a bit too far. I can't give any technical arguments
> against it, so I'm just looking for some perspective.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/f959c755-7a1a-499d-bad7-f0df8da44ce6%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Website: http://earl-of-code.com

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CAPiUgVfp%2BuoQB4o316ZKv%3DhK8D-aP0y3ZsGDaiO6g0PrVkqC4w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Opinions about aggressive reuse strategy for Jenkins pipelines

[David Karr]

I work for a very large organization, on a project with many teams, working 
on many microservices, all of which were generated from a home-grown 
application template system.  Most of the Java-based microservices have a 
very similar build process. In fact, most of the projects have an identical 
Jenkinsfile, with some small parameter value differences (which are set in 
a different system, not in the Jenkinsfile).

Over time, we've evolved the template and common features, including 
modularizing and cleaning up the Jenkinsfile.  A small shared library was 
developed, to hold some of the core function definitions, even though the 
basic structure of the build was still "in-line" in the Jenkinsfile.

The latest version of this template has taken reuse just about as far as it 
can go.  The entire Jenkinsfile for most projects will be a small 
properties block to override some defaults (that are not set in the other 
system), along with a single method call, including the shared library 
reference.  The shared library has now expanded to specify the entire build 
process, from that one method call.

There are a handful of services in the collection that will still have some 
custom changes (all in my domain), so I will reference a branch of the 
shared library in those Jenkinsfiles.  If my custom changes are eventually 
accepted as standard (many of them have been), then they will be merged to 
the master branch and I can change our Jenkinsfiles to use the master 
branch.

I'm describing all of this for background.  What I'm asking the community 
is, is this a reasonable strategy?  I can't help the feeling that this is 
taking reusability a bit too far. I can't give any technical arguments 
against it, so I'm just looking for some perspective.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/f959c755-7a1a-499d-bad7-f0df8da44ce6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.