Hello there, I'm working on Jenkinsfile-runner <https://github.com/kohsuke/jenkinsfile-runner> investigating possible use-cases. A major point I've noticed with this promising tool is that most Pipeline scripts will require some initial master setup (credentials, variables, node labels, tools, etc) so are not strictly portable. On the other hand, having worked on the Configuration-as-Code projet, I know a simple way to setup a jenkins master in a predictable and reproducible state. I'd like to combine both.
As a result, one could store in SCM - a Jenkinsfile to define the pipeline execution steps - a jenkins.yaml file to define the required master setup - a plugin.txt file to define the set of plugins required to run this pipeline (more on this later) I think this could be a nice driver for JCasC to embrace some extra use-cases, typically: how to smoothly integrate with a developer's workflow to *run arbitrary pipeline locally*. Part of this use-case is the ability to dry-run a pipeline <https://github.com/kohsuke/jenkinsfile-runner/issues/16>, which is a larger topic than just jenkinsfile-runner, but the later could be part of this story. Typically, it could offer simple ways to switch environment / credentials so that pipeline runs with test resources, and not production ones. Typically, I would expect we can run something like this : jenkinsfile-runner --config jenkins-dev.yaml Doing so would setup the headless jenkins master with development-only credentials, and environment variables to rely on test resources (a local web server as deployment target for sample), while the actual CI/CD master would run the same on production infrastructure. I also have in mind another possible usage : *one-shot jenkins masters* - let's call this "serverless jenkins <https://github.com/kohsuke/jenkinsfile-runner/issues/18>" for the buzzword. To avoid huge jenkins masters running thousand pipelines, we could rely on jenkinsfile-runner for pipeline execution, a front-end master being used for job configuration and web UI only. As a git event comes to jenkins infrastructure, a new serverless payload would be triggered (think : docker container running somewhere) to execute jenkinsfile-runner with adequate jenkins.yaml, so pipeline can run the same as if the front-end master had ran it by itself. This jenkins.yaml could be generated directly from master (this isn't yet implemented in JCasC, but definitively in the roadmap <https://github.com/jenkinsci/configuration-as-code-plugin/issues/65>) with the benefit it could be scoped to target job : this jenkins.yaml would only include credentials and other resources your job legitimately have access to, then when injected in transient, headless, jenkinsfile-runner's master, you gain physical security and can relax Pipeline's sandboxing. On pipeline completion, jenkinsfile-runner would then publish the build status on front-end master (something comparable to buildpublisher-plugin). About *plugins management*, we want this to be fully part of JCasC, and I have a pending PR <https://github.com/jenkinsci/configuration-as-code-plugin/pull/175> on this topic. From JCasC this is a chicken-egg issue, ad JCasC itself is a jenkins plugin. From jenkinsfile-runner this is way simpler to implement : as the setup take place before the headless master init sequence starts, we can download and install various plugins. As an initial state I've introduced support for plugins.txt file (the same one supported by jenkins official docker image <http://hub.docker.com/r/jenkins/jenkins>). Storing such a plugins list in SCM will let jenkinsfile-runner install the adequate plugins set on headless master to run the pipeline. Major drawback is this only supports standard update center as plugin source. A tighter integration with JCasC could be used to parse the plugins section from yaml configuration and get plugins from any update sites with adequate proxy settings. So, many challenges to address and potential extension for those tools. For me to get into the right direction, I would welcome any feedback, either on this thread or on github issues. -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJz%3D1t%3Du7v-T7tj3CypX5igdnen%2B%2BBvMEuBwmhOZQ5JTAFw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
