oh, and thanks for the tip on the readTrusted! Totally missed that one, and
yes, load is problematic because it requires the workspace (and thus a node)
On Wednesday, April 12, 2017 at 4:53:23 PM UTC-4, Patrick Wolf wrote:
>
> Feel free to open a JIRA ticket but I'm not a huge fan of this because it
> is counter to the KISS principle we wanted with Declarative and breaks the
> Blue Ocean editor. We have discussed having multiple "stages" blocks but
> rejected that because it quickly becomes needlessly complex without adding
> any use case coverage. IMO, having multiple "stages" makes much more sense
> than having multiple "pipelines" or else you will have to recreate all
> agent, environment, libraries, options, parameters etc for each pipeline
> and that leads to wanting those sections being DRY as well and Declarative
> pretty much falls apart completely.
>
> BTW, It is already possible to have multiple 'pipeline' closures in a
> single Jenkinsfile but they will be treated as parts of a whole Pipeline
> and this cannot be used in the editor. Because the Jenkinsfile is treated
> as one continuous Pipeline anything outside of the pipeline closures is
> interpreted as Scripted Pipeline. This means you can use 'if' blocks around
> the separate 'pipeline' blocks instead of using 'load' if you choose but
> keeping them in separate files makes maintenance easier, I think.
>
> if (BRANCH_NAME.startsWith("develop")) {
> pipeline { .... }
> }
>
>
> Also, it's worth noting that 'readTrusted' probably works better than
> 'load' because this takes the committer into account and it doesn't require
> a workspace.
>
>
> https://jenkins.io/doc/pipeline/steps/workflow-multibranch/#code-readtrusted-code-read-trusted-file-from-scm
>
> As for DRY stages there are several ways to accomplish this with Pipeline.
>
> 1. Shared Library and Resources - This is the preferred method of creating
> DRY routines
>
> You create a global variable that has all of the steps you want (with
> appropriate variable replacement for environment variables). You could have
> a build.groovy global variable in the /vars directory that does all of your
> build steps. Then the steps in your stage can be single line.
>
> Alternatively, you can store shell scripts in the /resources of your
> shared library and run those in your steps without having to duplicate
> anything:
>
> https://gist.github.com/HRMPW/92231e7b2344f20d9cc9d5f2eb778a54
>
> 2. You can define your steps directly in the Jenkinsfile at the top level
> either as strings or methods and simply call that method from with each
> pipeline.
>
> 3. You can define your steps in a configuration file as a property or yaml
> and load those files using the Pipeline utility steps plugin.
> https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Utility+Steps+Plugin
>
> To sum up, I think having different stages is worth discussing (it is not
> going to be implemented in the short term) but there are already many
> existing ways to make Pipelines DRY.
>
> On Tuesday, April 11, 2017 at 8:43:49 AM UTC-7, Kenneth Brooks wrote:
>>
>> TL;DR up front:
>> *As a user, I want to have a pipeline that performs specific pipeline
>> stages based on the branch. Recommendation: Put the when{} condition
>> outside the pipeline{} tag.*
>> *As a user, I want to declare my stages but have the implementation be
>> separate so that I can reuse them in multiple pipelines*.
>>
>> Currently the Declarative syntax has the ability to perform a stage
>> conditionally using 'when' but not a whole pipeline.
>> This leads to making the pipeline fairly inflexible and much harder to
>> read thru.
>>
>> Take for example:
>>
>> pipeline {
>>
>> stages {
>> stage('Build') {
>> when { branch "develop || master || feature"} // no the real syntax,
>> i know
>> steps { /* do some build stuff */ }
>> }
>>
>> stage('Scan') {
>> when { branch "master"}
>> steps { /* run static code analysis or other code scanning */}
>> }
>>
>> stage('Pull Request Build') {
>> when { branch "PR-*"}
>> steps { /* do a merge build stuff */ }
>> }
>>
>> stage('Dev Deploy') {
>> when { branch "develop || master"}
>> steps { /* deploy to dev */ }
>> }
>>
>> stage('Pull Request Deploy') {
>> when { branch "PR-*"}
>> steps { /* deploy to special PR sandbox */}
>> }
>> }
>> }
>>
>>
>> In this simple example, the following will happen, but it is extremely hard
>> to follow.
>>
>> Feature -> Build
>> Master -> Build, Scan, Dev Deploy
>> Develop -> Build, Dev Deploy
>> Pull Request -> Pull Request Build, Pull Request Deploy
>>
>> I would suggest we allow the when to be placed at the pipeline level
>> somehow.args
>>
>> pipeline('master') { // Just for naming
>> when { branch "master" }
>> stages {
>> stage('Build'){
>> steps { /* do some build stuff */ }
>> }
>> stage('Scan'){
>> steps { /* run static code analysis or other code scanning */}
>> }
>> stage('Dev Deploy'){
>> steps { /* deploy to dev */ }
>> }
>> }
>> }
>>
>> pipeline('develop') { // Just for naming
>> when { branch "develop" }
>> stages {
>> stage('Build'){
>> steps { /* do some build stuff */ }
>> }
>> stage('Dev Deploy'){
>> steps { /* deploy to dev */ }
>> }
>> }
>> }
>>
>> pipeline('pull request') { // Just for naming
>> when { branch "PR-*" }
>> stages {
>> stage('Pull Request Build') {
>> steps { /* do a merge build stuff */ }
>> }
>> stage('Pull Request Deploy') {
>> steps { /* deploy to special PR sandbox */}
>> }
>> }
>> }
>>
>> pipeline('feature') { // Just for naming
>> when { branch != "master || PR-* || develop" } // just do a build for any
>> 'other' branches, which would then include developer feature branches
>> stages {
>> stage('Build') {
>> steps { /* do some build stuff */ }
>> }
>> }
>> }
>>
>>
>> That, to me, is much cleaner. It is very easy to see exactly what each
>> pipeline is doing.
>> This brings one downside. The stage is repeated.
>> stage('Build') and stage('Dev Deploy') are the same impl, but I have to
>> write them 2 times.
>> I could create a global library, but then that has 2 other downsides. It is
>> no longer declarative syntax in the global library, the global library is
>> loaded external. I have to now go to a whole other file to see that
>> implementation.
>>
>> To keep things DRY I would also like to then see the stages treated as a
>> definition and and implementation.
>> Define the stages external to the pipeline, but pull them into each pipeline.
>>
>> This can optionally be done (like you'll see on the Pull Request stages).
>>
>> Here is what I believe the combination of the two would look like:
>>
>>
>> pipeline('master') { // Just for naming
>> when { branch "master" }
>> stages {
>> stage('Build')
>> stage('Scan')
>> stage('Dev Deploy')
>> }
>> }
>>
>> pipeline('develop') { // Just for naming
>> when { branch "develop" }
>> stages {
>> stage('Build')
>> stage('Dev Deploy')
>> }
>> }
>>
>> pipeline('pull request') { // Just for naming
>> when { branch "PR-*" }
>> stages {
>> stage('Pull Request Build') {
>> steps { /* do a merge build stuff */ }
>> }
>> stage('Pull Request Deploy') {
>> steps { /* deploy to special PR sandbox */}
>> }
>> }
>> }
>>
>> pipeline('feature') { // Just for naming
>> when { branch != "master || PR-* || develop" } // just do a build for any
>> 'other' branches, which would then include developer feature branches
>> stages {
>> stage('Build')
>> }
>> }
>>
>> /* Stage definitions below */
>> stage('Build'){
>> steps { /* do some build stuff */ }
>> }
>>
>> stage('Scan'){
>> steps { /* run static code analysis or other code scanning */}
>> }
>>
>> stage('Dev Deploy'){
>> steps { /* deploy to dev */ }
>> }
>>
>>
>> Is there a way to do this with the current declarative syntax?
>>
>> If not, what is the best way to get this into the declarative syntax? Open
>> jira enhancement requests?
>>
>>
>> What we've resorted to in the mean time (which still doesn't solve the DRY
>> part) is to have a Jenkinsfile that does the if logic and then loads a
>> specific pipeline (which has its own demons because the load evals the file
>> immediately and is holding onto a heavyweight executor the whole time).
>>
>>
>> if (env.BRANCH_NAME.startsWith("develop")) {
>> load 'develop-pipeline.groovy'
>> } else if (env.BRANCH_NAME.startsWith("master")) {
>> load 'master-pipeline.groovy'
>> } else if (env.BRANCH_NAME.startsWith("PR-")) {
>> load 'pull-request-pipeline.groovy'
>> } else {
>> load 'feature-pipeline.groovy'
>> }
>>
>>
>> Thanks,
>>
>> Ken
>>
>>
>>
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/caa3d201-2615-42f6-bd6b-97326472c353%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
