[
https://issues.apache.org/jira/browse/BIGTOP-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Boudnik updated BIGTOP-1368:
---------------------------------------
Assignee: Julien Eid
> Have Jenkins use Mesos-Jenkins plugin to create containerized build slaves
> --------------------------------------------------------------------------
>
> Key: BIGTOP-1368
> URL: https://issues.apache.org/jira/browse/BIGTOP-1368
> Project: Bigtop
> Issue Type: Improvement
> Components: Build
> Affects Versions: 0.7.0
> Reporter: Julien Eid
> Assignee: Julien Eid
> Fix For: backlog
>
>
> This ticket is about changing our Jenkins setup to do builds inside of
> containers so that we can auto-configure slaves and cleanup our environment
> as well as match containers that developers can use to build Bigtop.
> I recommend using the Jenkins Mesos plugin
> (https://github.com/jenkinsci/mesos-plugin) to start Jenkins slaves,
> orchestrate and run jobs in containers, and handle resource allocation to
> different slave containers that start up. Apache Mesos handles the launching
> of containers and limiting containers resources so that slaves won’t starve
> each other out for resources while the Jenkins Mesos plugin sits on top of
> Mesos and starts the slave.jar inside of the container which will then run
> the Jenkins building the BigTop package you want or doing whatever tests you
> need in the isolated environment. It would also not require much changing of
> our current Jenkins setup to get it integrated and build steps for most
> projects would be unchanged. You would just install mesos-master somewhere
> and then install mesos-slave on all of the servers that currently do job
> builds, and Mesos will handle using them all and spinning up the Jenkins
> slaves.
> While the above allows you to do BigTop builds in containers using Jenkins,
> we should also do some more things to make life easier. We can use the
> Jenkins Mesos plugin to run Docker in Docker (dind) so that inside of our
> Jenkins slave containers we can launch a Docker container inside of that to
> build Docker images for deployment. For example, you could have a Jenkins job
> that is kicked off inside of a container that then starts and builds a base
> Docker image that will later build a specific package on another job. This
> would allow our Jenkins setup to build and test our Puppet toolchain code and
> then produce images that would be used by other jobs for builds. This is
> documented and talked a lot about here:
> http://www.ebaytechblog.com/2014/04/04/delivering-ebays-ci-solution-with-apache-mesos-part-i/#.U7R4t3VdVhF
>
> As for matching builds on peoples computers, that gets pretty easy with
> Docker. You can just take the Jenkins Slave docker image and then run jobs
> against it without starting the slave.jar. So instead of docker run
> supervisord which you would do to get the Jenkins slave running, you can do
> docker run make hive-deb or whatever package you wanted. This would make
> peoples environments match our Jenkins environment exactly.
> I have an experimental git repo with some Dockerfiles that are a pretty good
> prototype of this future setup located at
> https://github.com/jeid64/bigtop-dockerfiles jenkins-docker/ has an example
> Ubuntu 12.04 Dockerfile that once built uses the Puppet manifests in
> bigtop_toolchain to setup the build environment. When run by Jenkins, Jenkins
> will start a slave inside of the container and the slave would handle all the
> build steps for that build. That image can also just be used on your desktop
> without needing a Jenkins master or using anything from Jenkins.
> Also in that repo is jenkins-dind/ which is based off of
> https://github.com/ahunnargikar/jenkins-dind multi-docker setup. This will be
> the image for dind image builds in Jenkins for build environments.
> I would love to head up this ticket and work on getting everything setup.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
