Re: Anyway to programatically update a Jenkinsfile?

[Torsten Reinhard]

we don´t use Jenkinsfile(s) at all - instead we generate all the jobs using 
the Job-DSL.
The jobs than either have the full (generated) code in it - or even better 
they just call some functions of a shared library.
Any modifications can be applied by a new version of the shared library - 
or by a commit that triggers and updates the JobGen job which will update 
all jobs.

It´s often a philosophical discussion about pros and cons of Jenkinsfile(s) 
- for real life projects it has a lot of disadvantages from my point of 
view.
It doesn´t solve your problem - but maybe there´s a way to move towards 
this approach of managing it?

Good luck, Torsten

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/0f0c008e-16da-428c-ab27-26ba7280b2e1%40googlegroups.com.

Re: Anyway to programatically update a Jenkinsfile?

[Martin d'Anjou]

Use a template engine to generate Jenkins files, I suggest the 
StringTemplate  template engine.
It's a purely declarative template.
There are examples in the intro doc 
 
that you can try in Groovy.

Martin

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/07665765-05d3-4f36-8083-b0b0c4697ab8%40googlegroups.com.

Re: Anyway to programatically update a Jenkinsfile?

[Tim Black]

I have to agree with Jeremy on this. Using a templating engine like m4 or 
jinja2 is superior to programmatic generation of groovy, if you need to do 
it on the fly. Why? Simplicity and Readability.

I inherited a build system that is based on using a multibranch pipeline 
with a Jenkinsfile that loads and executes a groovy script that's generated 
on the fly based on input from structured data from json files found 
alongside the Jenkinsfile in the repo. So, yeah, it's basically a 
hand-rolled solution to parametrizing a job. It sounds like that's 
basically what you're trying to accomplish as well, Jeff.

The part I don't like is the way we're generating the groovy code. We've a 
library of python code that uses classes and a streaming approach to 
outputting all the syntax required to implement the pipeline. First of all, 
there's a high learning curve to using this approach - when trying to 
determine what the groovy output will look like you have to run the code 
and look at the output, which is a whole software development process in 
and of itself. When making changes, instead of using a template with 
variable substitution and loops perhaps, with the programatic approach, you 
have to write (and test and debug) code to make the groovy code you want. A 
template already resembles what the final output (groovy) will look like, 
so it's quite intuitive to work on. Also, every time you need to use a new 
groovy construct, with the programatic approach, you have to add support 
for it in your code gen library. With the template you just write it, based 
on the pipeline syntax documents. This brings up another reason why 
templated approach is superior: Programatic approach obfuscates the details 
of the language  from the code gen developer. Looking back and forth 
between the jenkins pipeline syntax docs and a template is straightforward, 
not so with a library of classes defined to generate groovy.

So, that's my $0.02. I plan to refactor our code gen library to use jinja2 
templating to make our pipelines more intuitive and easier to work with.

Another thing related to keep in mind is that, as far as I can tell, the 
"load" command, which is the mechanism you would use to actually run your 
generated pipeline script, is designed to work with Scripted Pipeline, not 
Declarative. This distinction is very poorly documented by Cloudbees, IMO. 
Please chime in if you have more info on dynamic generation and running of 
pipeline code.

On Tuesday, October 1, 2019 at 8:43:25 AM UTC-7, Jeff Sault wrote:
>
> Does anyone know of any libraries/tools that will allow me to 
> add/remove/update sections of a Jenkinsfile programatically? I have a load 
> of different projects which I need to update to include some new mandatory 
> parameters. I'd like to go down the 'shared library' route but in this 
> instance its not really possible. Parsing the jenkinsfile is non-trivial 
> but I assume theres something in groovy/jenkins land which can already do 
> the job.
>
> Thanks
> Jeff
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/d77f7156-b8b9-4b65-9959-1aad8a39f2a5%40googlegroups.com.

Re: Anyway to programatically update a Jenkinsfile?

[Slide]

Pipeline scripts are a subset of groovy, so you may want to look in the
direction of the available groovy AST modification methods. I am not sure
how well it will work on declarative (though it should), but it's the best
bet I believe.

On Wed, Oct 2, 2019 at 12:07 AM Jeff Sault  wrote:

> Thanks but i dont think it suits in my case, i need to update a bunch of
> jenkinsfiles owned by other teams. Surely theres a way to parse a
> Jenkinsfile into an object, make changes and then write the updated
> Jenkinsfile back out to the filesystem?
>
> On Tuesday, October 1, 2019 at 4:59:44 PM UTC+1, jeremy mordkoff wrote:
>>
>> I use a templating tool (m4) so I can build up a groovy script using
>> macros. In the first stage of my Jenklinsfiie, I call a bash script that
>> generates a groovy script, which I then load and execute. The loaded script
>> has N stages. Note that if the ordering of your stages changes, jenkins
>> will stop showing history, so I try to always add stages at the end.
>>
>>
>>
>> On Tuesday, October 1, 2019 at 11:43:25 AM UTC-4, Jeff Sault wrote:
>>>
>>> Does anyone know of any libraries/tools that will allow me to
>>> add/remove/update sections of a Jenkinsfile programatically? I have a load
>>> of different projects which I need to update to include some new mandatory
>>> parameters. I'd like to go down the 'shared library' route but in this
>>> instance its not really possible. Parsing the jenkinsfile is non-trivial
>>> but I assume theres something in groovy/jenkins land which can already do
>>> the job.
>>>
>>> Thanks
>>> Jeff
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/db51294d-c934-47bf-87f7-a3f113cf618d%40googlegroups.com
> 
> .
>


-- 
Website: http://earl-of-code.com

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CAPiUgVfef6KVgshTrvR6DtaOwZ20hz431%3DSKC%3DWLOi0u0Fe4FQ%40mail.gmail.com.

Re: Anyway to programatically update a Jenkinsfile?

[Jeff Sault]

Thanks but i dont think it suits in my case, i need to update a bunch of 
jenkinsfiles owned by other teams. Surely theres a way to parse a 
Jenkinsfile into an object, make changes and then write the updated 
Jenkinsfile back out to the filesystem?

On Tuesday, October 1, 2019 at 4:59:44 PM UTC+1, jeremy mordkoff wrote:
>
> I use a templating tool (m4) so I can build up a groovy script using 
> macros. In the first stage of my Jenklinsfiie, I call a bash script that 
> generates a groovy script, which I then load and execute. The loaded script 
> has N stages. Note that if the ordering of your stages changes, jenkins 
> will stop showing history, so I try to always add stages at the end. 
>
>
>
> On Tuesday, October 1, 2019 at 11:43:25 AM UTC-4, Jeff Sault wrote:
>>
>> Does anyone know of any libraries/tools that will allow me to 
>> add/remove/update sections of a Jenkinsfile programatically? I have a load 
>> of different projects which I need to update to include some new mandatory 
>> parameters. I'd like to go down the 'shared library' route but in this 
>> instance its not really possible. Parsing the jenkinsfile is non-trivial 
>> but I assume theres something in groovy/jenkins land which can already do 
>> the job.
>>
>> Thanks
>> Jeff
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/db51294d-c934-47bf-87f7-a3f113cf618d%40googlegroups.com.

Re: Anyway to programatically update a Jenkinsfile?

[jeremy mordkoff]

I use a templating tool (m4) so I can build up a groovy script using 
macros. In the first stage of my Jenklinsfiie, I call a bash script that 
generates a groovy script, which I then load and execute. The loaded script 
has N stages. Note that if the ordering of your stages changes, jenkins 
will stop showing history, so I try to always add stages at the end. 



On Tuesday, October 1, 2019 at 11:43:25 AM UTC-4, Jeff Sault wrote:
>
> Does anyone know of any libraries/tools that will allow me to 
> add/remove/update sections of a Jenkinsfile programatically? I have a load 
> of different projects which I need to update to include some new mandatory 
> parameters. I'd like to go down the 'shared library' route but in this 
> instance its not really possible. Parsing the jenkinsfile is non-trivial 
> but I assume theres something in groovy/jenkins land which can already do 
> the job.
>
> Thanks
> Jeff
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/7a2da374-5734-4eb9-a4a2-fed3254d8ca4%40googlegroups.com.