Is there a way to basically "unzip" the *jenkins.war* so that the plugins, workflow-libs, and other parts can be configured before actually running the service?
On Wednesday, August 17, 2016 at 9:36:08 AM UTC-5, Jason Kulatunga wrote: > > Hey, > Thanks for all the help guys. > I slept on this idea for a few days because, to be honest I really didn't > want to write my own package manager > <https://medium.com/@sdboyer/so-you-want-to-write-a-package-manager-4ae9c17d9527#.jieuao7e5> > and > re-invent the wheel. I took a step back and looked at how Jenkins solved > this problem for Plugin developers, and I think that we could just > piggy-back on top of what they use > <https://github.com/jenkinsci/gradle-jpi-plugin>. > > Basically what I've done is specify the plugins I want to install in a > build.gradle file on my Jenkins server. The build.gradle file lets me > specify exactly what versions of the plugins I want for some, and get the > latest for the rest. My install task then goes and copies just the runtime > hpi files to the $JENKINS_HOME/plugins folder (after clearing out whatever > is in there). After restarting my Jenkins server, all plugins are > installed, with the correct versions. > > I've included a plugin management section in my blog post: You Don't Know > Jenkins - Part 1 > <http://blog.thesparktree.com/post/149039600544/you-dont-know-jenkins-part-1> > which > goes into more detail on how it all works, and includes an example > build.gradle file. > > Things to note: > - The plugin.lock file isn't perfect, its just a STDOUT redirect of > `gradle dependencies` which is great for visually checking which versions > are installed, but committing it to git gets you nothing, subsequent > installs wont be locked to the same transient dependencies. I think I can > solve this by using > https://github.com/nebula-plugins/gradle-dependency-lock-plugin > - Since the build.gradle file uses repo.jenkins-ci.org instead of > updates.jenkins-ci.org it does pick up the occassional beta/alpha version > that gets pushed to the releases repo by developers. I'm working to fix > this using a filter in the gradle dependency solver configuration. > > > > On Thursday, August 11, 2016 at 6:03:12 PM UTC-7, Michael Kobit wrote: >> >> We are looking at doing something similar (actually talking about this >> with colleagues today). The idea is to basically build an immutable Jenkins >> instance that can't be modified. Or at least severely limit any kinds of >> modifications to it so that we have an easily deployable "Jenkins as a >> service". >> >> I've looked at possibly doing an "unpack and install" execution with the >> *jenkins.war >> *, but it doesn't look like an easy route. The other pain-point I see is >> effectively treating the correct files as "data" that should be persisted >> over time, rather than at "Jenkins build time". I am considering trying out >> the Docker-type approach. I think for plugin resolution, we are probably >> going to have to go the route that you are talking about for doing the >> resolution ourselves. >> >> For security type issues, I think we could still handle it with the >> Docker approach. Build whatever restrictions into the next "immutable" >> image and making that deployable. Then, we can have a "staging" area and >> easily rollback if we effectively control all the things we need to >> control. We are experimenting with pipelines right now, and are waiting to >> see how https://issues.jenkins-ci.org/browse/JENKINS-33507 will work for >> us to get as much of the job configuration out of Jenkins as possible. >> >> We are still in the brainstorming phase, so I'm interested to see who >> else has ran into this and what they have done. >> >> On Thursday, August 11, 2016 at 5:47:45 PM UTC-5, Jason Kulatunga wrote: >>> >>> Hey, >>> Thanks for all the feedback :) >>> >>> @Daniel Beck: >>> Yup, I'm familiar with the limitations of the >>> https://updates.jenkins-ci.org/current/update-center.json file. Thats >>> why I'm thinking of creating a plugin/dependency resolution system that >>> will have to directly download the specific version of a plugin file from >>> update site folder structure >>> https://updates.jenkins-ci.org/download/plugins/*/ or use >>> https://updates.jenkins-ci.org/latest/ >>> if no version restriction is found. >>> >>> I wasn't aware that pinning was pointless in 2.x so that'll be an >>> interesting problem to deal with. It seems that I'll have to restrict all >>> access to the UpdateCenter for idea #1, or do a hybrid approach with a >>> UpdateCenter subclass as well. >>> >>> @Baptiste Mathus >>> Unfortunately just using an image with locked plugins isn't a long term >>> solution, because we'll have to occasionally update our Jenkins due to >>> required security updates in plugins or the main application. So being able >>> to update plugins, creating a new *.lock file, test the plugin interactions >>> and deploying the *.lock file to existing Jenkins servers is a requirement. >>> >>> I was hoping to stay away from a hybrid approach that used both an >>> executable and a subclass as it makes development and deployment more >>> complicated, decreasing adoption with Jenkins users. >>> >>> Honestly the goal was to create a tool like Bundler/Pip which would just >>> work out of the box for 99% of use cases. >>> >>> Are there other people experiencing the same issue? I'm more than happy >>> to create my own open source solution, but I'd love to base it on an >>> existing (even unmaintained) project. >>> >>> -Jason >>> >>> >>> On Thursday, August 11, 2016 at 4:51:07 PM UTC-4, Baptiste Mathus wrote: >>>> >>>> IMO a Docker image with the right set of plugins you've tested, plus >>>> the security config you're talking about about forbidding any upgrade >>>> would >>>> seem a simpler way. And probably it would your life simpler if you somehow >>>> have to support all those different instances which can currently be >>>> actually quite different. >>>> >>>> HTH >>>> >>>> Le 11 août 2016 3:14 PM, "Jason Kulatunga" <[email protected]> a >>>> écrit : >>>> >>>> Hey Jenkins-Users, >>>> >>>> I manage almost a dozen Jenkins servers and our team has been having >>>> some issues with plugin management: such as locking our new installations >>>> to known working versions of some troublesome Jenkins plugins. >>>> We use chef + Jenkins DSL to completely automate the initial >>>> installation of Jenkins, but we're not exactly thrilled with the way the >>>> Chef cookbook handles plugin installation and we've also verified that >>>> 'installNecessaryPlugins' does not actually respect the version parameter. >>>> >>>> curl -XPOST >>>> http://192.150.23.105:8080/pluginManager/installNecessaryPlugins -d >>>> '<install plugin="[email protected]" />' >>>> >>>> As such I've started looking into alternative means of locking plugins >>>> in an automated way during installation and I've come up with the >>>> following >>>> ideas: >>>> >>>> # An External Dependency Management Tool, eg Bundler, Pip, Berkshelf >>>> Basically an executable that would: >>>> >>>> 1. retrieve a list of all plugins installed in a specific Jenkins >>>> server using the API, and add them to a dependency graph (with >>>> metadata: >>>> installed, pinned, enabled, version) >>>> 2. look for a dependency config file (like Gemfile, Berksfile, >>>> requirements.txt) >>>> 3. iterate through all the uninstalled plugins in the dep config >>>> file and add them (and their dependencies) to the dependency graph >>>> 4. solve the graph by ensuring that no pinned/locked version >>>> conflicts occur. >>>> 5. download all uninstalled plugins directly from >>>> https://updates.jenkins-ci.org/ >>>> 6. using the Jenkins api, pin any version locked plugins specified >>>> in the dependency config file. >>>> 7. write the solved dependency graph to the filesystem (eg >>>> Berksfile.lock, Gemfile.lock) (and use it for any subsequent installs >>>> if no >>>> plugins have changed) >>>> 8. disable all permissions to the update center in Jenkins so no >>>> users enable/update plugins manually. >>>> >>>> # UpdateCenter Override >>>> >>>> 1. subclass the default Jenkins UpdateCenter, and tell Jenkins to >>>> use it using a JVM property >>>> 2. override the UpdateCenter.InstallationJob constructor to >>>> download the plugin version specified from the dependency config lock >>>> file >>>> if it exists or install like normal and then generate/update a >>>> dependency >>>> config lock file with every operation. >>>> 3. listen to the pin event in the PluginCenter and update the >>>> dependency config lock file. >>>> >>>> I'm not sure if anyone has done something similar but I wanted to get >>>> some feedback before I spent too much time investigating either idea. >>>> >>>> Any and all feedback is welcome >>>> >>>> -Jason >>>> Build Automation Engineer >>>> Adobe >>>> >>>> >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Jenkins Users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/jenkinsci-users/2d4f0e32-7d6a-4159-9635-51df7ff83643%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/jenkinsci-users/2d4f0e32-7d6a-4159-9635-51df7ff83643%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f3e2ca52-d6ff-4dc1-b5fe-5085764dddc0%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
