On Sun, Mar 4, 2018 at 8:13 AM <[email protected]> wrote:
> I think Jenkinsfile Runner brings a lot of opportunities for pipeline
> developers. The most obvious ones to me are
>
> 1. Pipeline development (Jenkinsfile)
> 2. Shared library development
>
> *Pipeline development*
>
> Right now (as described by others in this thread) pipeline development is
> either a loop of committing / fixing pipelines on production Jenkins, using
> pipeline replay on production Jenkins or setting up a local instance
> manually.
>
> With Jenkinsfile Runner we can get faster feedback without polluting our
> commit or Jenkins build history and don't have to set up a local instance
> manually.
>
Right. I think we all get this basic picture. Details are where things get
interesting!
*Shared library development*
>
> Shared library development right now works much in the same as pipeline
> development, except that you have the library code and another (often
> production) Jenkinsfile to maintain, in order to try out (as opposed to
> automatically test) your Jenkinsfile.
> For shared libraries, we thankfully already have JenkinsPipelineUnit, that
> makes it easier to implement some tests. However, (as also mentioned by
> others in this thread) this is unit testing only. It mocks most of our
> environment. Often, green unit tests mean nothing for productive use of our
> share library. I even gave up test-driven development for shared libraries,
> in favor of 90s try-and-error-style programming. Because most of the time
> when trying the library with green unit tests in production, it turns out
> that the real Jenkins environment has some restriction that is beyond the
> scope of JenkinsPieplineUnit (e.g. sandboxing).
>
Worst thing about the current state is that we don't have reliable
> regression tests. A change in shared library with green unit tests is far
> from convincing me that the library will continue to work in production.
>
> With Jenkinsfile Runner we could write small Jenkinsfiles within the
> shared library repo's test folder and run them from the Jenkinsfile of the
> shared lib. Pretty much in the same way as we use Maven Invoker Plugin (as
> mentioned by Jesse) when developing maven plugins.
>
OK, thank you, this is really helpful. I think maven invoker plugin analogy
is a great one for somebody like me. So from this perspective, mocking
isn't really high on the priority.
>
> *A first approach to shared library integration testing with Jenkinsfile
> Runner*
> My naive first approach was to build a Docker Image that contains
> Jenkinsfile Runner and all default plugins.
>
> docker run -v~/src/it/myfunction:/workspace schnatterer/jenkinsfile-runner
> :1.0-SNAPSHOT-2.108 /workspace
>
> runs the ~/foo/Jenkinsfile using Jenkinsfile Runner with all default
> plugins of Jenkins 2.108.
>
> My idea was to eventually do the same in Jenkinsfile of the shared lib
> like so (not tested)
> docker.image('schnatterer/jenkinsfile-runner:1.0-SNAPSHOT-2.108').inside {
>
> sh 'jenkinsfile-runner /app/jenkins /app/plugins src/it/myfunction'
> }
> or
> sh 'docker run --rm -v $(pwd):$(pwd) src/it/myfunction'
>
So if this is the intended use case, then I have another idea. Instead of
running Jenkinsfile in this CLI and try to emulate your Jenkins instance as
closely as possible (in terms of configuration), we could just run
Jenkinsfile in the current Jenkins, in a place that nobody else can see.
For example, packaged as CLI, you'd run it like:
$ java -jar jenkins-cli.jar -s https://jenkins.example.com/ run-pipeline
./src/it/myfunction
Or from within Jenkinsfile, could be something like:
pipeline.sharedlibrary.test './src/it/myfunction'
... and this would completely bypass the challenge of trying to replicate
the Jenkins configuration.
> It turned out that there a two major problems:
>
> 1. There's no way to add non-default Jenkins plugins.
> My local test for cloudogu/ces-build-lib
> <https://github.com/cloudogu/ces-build-lib> failed because there was
> no GitHub Groovy Libraries plugin.
> Here, my hope is that Configuration-as-code plugin might help automate
> loading plugins.
>
> Yeah, I'm really looking forward for CaC project to fill this gap, and I
know they are working on it.
<https://github.com/jenkinsci/jep/pull/59/files>
>
> 1. There's still no way to load a "local" shared library from the same
> repo. So, we still would have to find a way to configure the shared library
> in our Jenkinsfile Runner.
> Loading local shared libraries has already been discussed here
> <https://github.com/jenkinsci/workflow-cps-global-lib-plugin/pull/37>
> and there <https://stackoverflow.com/q/46213913/1845976>.
>
> Good point.
> Once those issues are solved, we'll have a very basic way of automating
> integration tests for shared libraries by executing IT Jenkinsfiles from
> the shared libraries pipeline and failing the build if the IT fails.
>
That'd be really cool, isn't it!?
> Of course, this would be very basic testing. For more sophistiated testing
> we would want to
>
>
> - trigger the ITs from maven or gradle,
> - use asserts,
> - get the results as JUnit XML.
>
> So, yes, we're not there yet. But we now have a foundation to build all
> this upon.
>
asserts would already work, right? It seems like we can do that relatively
easily on its own.
But with that aside, I think I understand the picture in your head. You are
really approaching it like Maven plugin development. Indeed this would help
illustrate relative strength of both JenkinsPipelineUnit and "integration
tests"
>
> Thanks for that & best regards,
>
> Johannes
>
>
> Am Donnerstag, 1. März 2018 20:23:15 UTC+1 schrieb Kohsuke Kawaguchi:
>>
>> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution
>> as a command line tool. The intend use cases include:
>>
>> - Use Jenkins in Function-as-a-Service context
>> - Assist editing Jenkinsfile locally
>> - Integration test shared libraries
>>
>> Over the past year, I've done some deep-dive 1:1 conversations with some
>> Jenkins users and I felt something like this might move the needle for them
>> in an important way.
>>
>> I'd love to hear any reactions on your side. Could something like this be
>> important for you, does it miss any key points for you? If you mentally
>> picture a perfect version of this, what would that do, and how would you
>> use?
>>
>> Let me know!
>>
>> --
>> Kohsuke Kawaguchi
>>
>
>
> Am Donnerstag, 1. März 2018 20:23:15 UTC+1 schrieb Kohsuke Kawaguchi:
>>
>> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution
>> as a command line tool. The intend use cases include:
>>
>> - Use Jenkins in Function-as-a-Service context
>> - Assist editing Jenkinsfile locally
>> - Integration test shared libraries
>>
>> Over the past year, I've done some deep-dive 1:1 conversations with some
>> Jenkins users and I felt something like this might move the needle for them
>> in an important way.
>>
>> I'd love to hear any reactions on your side. Could something like this be
>> important for you, does it miss any key points for you? If you mentally
>> picture a perfect version of this, what would that do, and how would you
>> use?
>>
>> Let me know!
>>
>> --
>> Kohsuke Kawaguchi
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/fcdcb7e4-f6ca-4ec5-bf7d-0c9ca32618ee%40googlegroups.com
> <https://groups.google.com/d/msgid/jenkinsci-dev/fcdcb7e4-f6ca-4ec5-bf7d-0c9ca32618ee%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
Kohsuke Kawaguchi
--
You received this message because you are subscribed to the Google Groups
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4wqSeMyg9sY0G1xC9pEtgGqDvU-w42VrLgPzYRYNKavZg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
