On Tue, Mar 6, 2018 at 2:57 AM Bill Dennis <bill.den...@gmail.com> wrote:
> Thanks Kohsuke, I tried to give some answers to your questions inline > below, if I didn't mess up the reply.. > > Bill > > On Friday, 2 March 2018 17:57:24 UTC, Kohsuke Kawaguchi wrote: > >> >> >> On Fri, Mar 2, 2018 at 9:26 AM Bill Dennis <bill....@gmail.com> wrote: >> >>> Hello Kohsuke - >>> >>> I am a developer using Jenkins pipeline quite lot where I work. >>> >>> We are using Jenkins pipelines in two scenarios: >>> >>> - For CI building and testing some of our internal components (what >>> Jenkins is traditionally used for) >>> - For running / orchestrating complex automation processes (so >>> Jenkins is talking to some external systems using SOAP / REST etc) via >>> tooling and even directly via REST using plugins. >>> >>> I have mostly used JenkinsPipelineUnit for testing / validation of the >>> pipelines and I have been looking into the direct / live approach that Oleg >>> demonstrated (running Jenkins locally in Docker and getting the pipeline >>> being developed direct from a host file system volume mount). >>> >>> I think Jenkinsfile Runner would be really useful for developers who >>> don't need or want the overhead of developing tests with >>> JenkinsPipelineUnit. I have worked with some developers wanting to develop >>> Jenkinsfiles for their CI process and the main problem is knowing if the >>> Jenkinsfile will work when they commit it to the repo. They go round this >>> loop of commit / fix running in the production Jenkins or using the Jenkins >>> pipeline "replay" feature. It can be a painful process if you are not >>> familiar with Jenkins pipeline and Groovy syntax! >>> >> >> This kind of context is really helpful. Thank you! >> > > Happy to feedback. Thanks for Jenkins and pipeline as code, it helped me > deliver some projects with Jenkins in a way that I thought would not be > possible a few years ago. > My pleasure, and really the portion I can take credit for is getting smaller every day! > > >> I think some things to consider are: >>> >>> - How does the Jenkins Runner replicate the agents / slaves identifiers >>> on the target Jenkins? >>> - How to deal with tooling on the target Jenkins (custom tools, JDKs, >>> Gradle, etc)? >>> >> >> Right, I guess your point is that Jenkinsfile Runner should aim to run >> Jenkinsfile in much more realistic setup, and that doesn't stop at using >> real Jenkins and real Pipeline plugins, but it also needs to include other >> configurations of Jenkins. I think Jesse made a similar observation. I have >> a few thoughts: >> >> - Configuration-as-code >> <https://github.com/jenkinsci/configuration-as-code-plugin> could >> play a role here in terms of letting people define the configuration of >> Jenkins once and use it both in production and in setup like Jenkinsfile >> Runner >> - I'm a fan of making Jenkinsfile itself more portable. For example, >> if people are already in the mode of using docker images to run builds in, >> then more of the toolings would be packaged in there, and it should allow >> Jenkinsfile Runner to run your project in much the same way as your >> production Jenkins. I'm curious how much of this is already reality vs >> ideal that people are working toward. >> >> Yes, all of this. I have often thought that we need something like > declarative pipeline for the configuration of Jenkins as code instead of > going into all those web config pages. Jenkins master as Docker container > seems good. In our environment we are not currently using Docker but I have > seen that that is the way to go. Getting a larger organisation to adopt the > right technology and the associated costs of that is the challenge, so we > remain using traditional Jenkins slaves and tooling methods. Hopefully > Dockerised soon. We do use Jenkins Enterprise from CloudBees. > "something like declarative pipeline for the configuration of Jenkins as code instead of going into all those web config pages" --- please do check out the linked configuration-as-code effort. I think you'll love it. And thanks for the explanation of the state of your current Jenkins master. I think the perfect Jenkinsfile Runner for me would provide: >>> >>> - Somehow capture the plugins, tooling and agents on our production >>> Jenkins >>> - Validate the Jenkinsfile pipeline syntax >>> >> >> I think this is already happening as a result of actually running the >> pipeline -- one of the virtue of actually using the real pipeline plugins >> to run! >> >> >>> - Validate the Jenkinsfile against the plugins and agents / tooling >>> (fail if it refers to some tool or agent not configured for example). >>> - Run the Jenkinfile in some sort of "no-op" mode : what would it do if >>> I ran it, without actually doing anything >>> >> >> This one is interesting. I assumed JenkinsPipelineUnit does this pretty >> well, though. Can you tell me more about this? I'm imagining you'd want to >> be able to selectively mock out some steps (e.g., when Jenkinsfile gets to >> sh "./deploy.sh" don't actually do it and pretend that it succeeded) but >> more details would be helpful. >> > > Yes, in JenkinsPipelineUnit I pretty much mock out everything like calls > to tools. Because pipeline is based on Groovy I thought I could do some > unit tests using Spock and Groovy for pipelines but then I discovered it > was already done and shared as JenkinsPipelineUnit, so hat tip to Ozan > there. So the Spock unit tests I write are confirming the execution of the > tools is correct but not actually running them. I also write > JenkinsPipelineUnit test code for my share library code as well confirming > the behaviour, so actually all my use of special tooling is captured in > libraries as DSL like constructs. A great feature of JenkinsPipelineUnit is > that it can generate the execution callstack of the "mocked" pipeline run. > These callstacks can be stored as files and JenkinsPipelineUnit tests can > assert the execution callstack of a test run against a file committed as > part of the tests. I do this for all my pipelines and share library units. > So the callstacks actually capture the execution of tooling with > commandline params etc and any time I change the pipeline code I can see > what is changed by diffing the callstack file changes. Another thing you > can do with this is run the pipeline jobs with different parameters in the > tests (Spock parameterised tests) and by comparing the callstack files can > see how the pipeline behaviour differs between these runs and assert the > correct behaviour. All this without having to hit a real Jenkins server and > wait for long running processes to complete! So this is all fine and you > can write tests for the pipelines and check the use of tooling etc, but > when you come to run the pipeline on the actual server, the tools or agents > / slaves labels are not configured as expected for the fully tested > pipeline. So this is where something like Jenkins Runner will be good to > use. My tests are based on code methods in the project I have here: > https://github.com/macg33zr/pipelineUnit > So just to make sure I understand you correctly, given all that you do with JenkinsPipelineUnit, the point of "integration test" for you is to actually run in real Jenkins, in order to verify that agent labels exist, those agents do have the tools that your 'sh' step uses, etc. - Actually run the Jenkinsfile locally so I can know it works completely >>> before committing to source control. >>> >> >> Yeah, this was the first goal for me. >> >> >>> - Run the Jenkinsfile on the target Jenkins master server using the >>> resources of that server (so know it works on the server). >>> >> >> This got me thinking that maybe all I needed was a Jenkins CLI command >> that behind the scene creates a temporary/hidden job on the target Jenkins >> master and run the Pipeline. IOW, keep the same development flow as >> Jenkinsfile Runner today, but don't run Jenkins locally, just do it on your >> actual Jenkins. >> > > Yes, this would be great, an anonymous job and just pull back the output > and console log. A plugin for IntelliJ or whatever IDE using this would be > even better :-) > Great, between your feedback and what I think Johanness is saying, I'm starting to picture the next POTD. Hope that helps! >>> >>> Bill Dennis >>> >>> >>> On Thursday, 1 March 2018 19:23:15 UTC, Kohsuke Kawaguchi wrote: >>>> >>>> Jenkinsfile Runner is an experiment to package Jenkins pipeline >>>> execution as a command line tool. The intend use cases include: >>>> >>>> - Use Jenkins in Function-as-a-Service context >>>> - Assist editing Jenkinsfile locally >>>> - Integration test shared libraries >>>> >>>> Over the past year, I've done some deep-dive 1:1 conversations with >>>> some Jenkins users and I felt something like this might move the needle for >>>> them in an important way. >>>> >>>> I'd love to hear any reactions on your side. Could something like this >>>> be important for you, does it miss any key points for you? If you mentally >>>> picture a perfect version of this, what would that do, and how would you >>>> use? >>>> >>>> Let me know! >>>> >>>> -- >>>> Kohsuke Kawaguchi >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Developers" group. >>> >> To unsubscribe from this group and stop receiving emails from it, send an >>> email to jenkinsci-de...@googlegroups.com. >> >> >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/jenkinsci-dev/b475ecc2-2807-47c7-809f-5e6c1a51d2e0%40googlegroups.com >>> <https://groups.google.com/d/msgid/jenkinsci-dev/b475ecc2-2807-47c7-809f-5e6c1a51d2e0%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> Kohsuke Kawaguchi >> > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jenkinsci-dev+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-dev/c1ae5ba9-d54d-4c87-ad3d-d7b1ae004148%40googlegroups.com > <https://groups.google.com/d/msgid/jenkinsci-dev/c1ae5ba9-d54d-4c87-ad3d-d7b1ae004148%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- Kohsuke Kawaguchi -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4z6CtczryLyAZhiuQ-dOFZ5rZNUG__RZqZM25utNs5ntQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.