Hey, You should probably start with the general things like https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds https://wiki.jenkins-ci.org/display/JENKINS/Consideration+for+Large+Scale+Jenkins+Deployment or https://jenkins.io/doc/book/architecting-for-scale/ and Googling about "Scaling jenkins" to get a grasp about the domain.
About your question, one of the first things to know/do when scaling Jenkins, is to not run builds on the Master. Basically, you generally set the executor to 0, (or a very very low number and drastically select the ones you allow running there). About using Docker, using a Docker Swarm cluster as a build cluster, used by the master like it has virtual infinite capacity works well (using the docker-plugin), though probably not the one with the most Docker spirit way (see docker-slaves-plugin, though not the same goal). <shameless-plug>If you speak french, I demoed that using basically https://github.com/batmat/jez at the ToulouseJUG/JAM here <https://www.youtube.com/watch?v=AB-OBl1hAPs>.</shameless-plug> HTH 2016-11-25 17:13 GMT+01:00 Eddard <[email protected]>: > Hello, > > the software engineering team i work in is putting up a new ci/cd platform > based on jenkins which will orchestrate the continuous delivery > pipeline for more than 50 micro-services in a near future. We already have > 10 of them being built and deployed through that platform. > > we have come to the realization that out setup light not be scalable and > are looking for an alternative architecture/setup for the platform. > > > - Right now we have a bare meta machine with a docker-engine on it > which allow ton a docker container with the master jenkins in it. > - when we run a pipeline build, we launch a standard ubuntu container > as a slave agent which runs on the same docker host as the jenkins master, > this slave container > - the ubuntu container slave has a docker and docker-compose installed > which in turn are used to launch container on the same docker-host of the > master to run all the tasks of the pipeline. > > This is not very optimal and we are looking for alternatives, if you have > a ci-platform setup based on jenkins master running in a container and has > scaled to run many many builds would you please share your experiences? and > what would > > you consider an optimal and scalable jenkins in docker setup/architecture? > > > Thanks in advance > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/jenkinsci-users/b12830f9-c5cf-411f-a489-5cf2ada3aafd%40googlegroups. > com > <https://groups.google.com/d/msgid/jenkinsci-users/b12830f9-c5cf-411f-a489-5cf2ada3aafd%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS6Ke8-ZA08STSsdYo3DQAFf79zsr6dCbvBL%2BU1etbZ2OQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
