[
https://issues.apache.org/jira/browse/BIGTOP-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14055737#comment-14055737
]
Roman Shaposhnik commented on BIGTOP-1368:
------------------------------------------
Given that we're talking relative merits of Jenkins plugins, I'm sure [~abayer]
will help set us straight ;-) With that, here's a few comments:
bq. CoreOS is in my opinion not production
The only reason I'm using it is because it is convenient -- anything that gets
me Docker 1.0 would do. IOW, you can replace CoreOS with whatever distro you
want -- Jenkins EC2 plugin will spin it up just the same.
bq. The way it would work is you would spin up EC2 instances to be slave hosts
with a very minimal install of CentOS or Ubuntu or something, the distro for
the slaves doesn't matter. Mesos-slave would be installed on that EC2 instance
that was spun up and it would connect to the Mesos-master.
This is the bit I'm not following -- once Jenkins spins up EC2 slaves we're
done. We can simply run $ docker ... builds on them. Why an extra layer of
Mesos?
bq. From what I've seen, there is not much work done to have a Jenkins plugin
to start build slaves on CoreOS
We're currently using Jenkins EC2 plugin which is very much maintained and
hasn't failed us much.
bq. Mesos-slave would be installed on that EC2 instance that was spun up and it
would connect to the Mesos-master.
And that is the bit that worries me: who will be maintaing Mesos-master and
making sure it all works? This is an extra bit of work that I'm trying to avoid
by relying on EC2 plugin.
> Have Jenkins use Mesos-Jenkins plugin to create containerized build slaves
> --------------------------------------------------------------------------
>
> Key: BIGTOP-1368
> URL: https://issues.apache.org/jira/browse/BIGTOP-1368
> Project: Bigtop
> Issue Type: Improvement
> Components: Build
> Affects Versions: 0.7.0
> Reporter: Julien Eid
> Assignee: Julien Eid
> Fix For: backlog
>
>
> This ticket is about changing our Jenkins setup to do builds inside of
> containers so that we can auto-configure slaves and cleanup our environment
> as well as match containers that developers can use to build Bigtop.
> I recommend using the Jenkins Mesos plugin
> (https://github.com/jenkinsci/mesos-plugin) to start Jenkins slaves,
> orchestrate and run jobs in containers, and handle resource allocation to
> different slave containers that start up. Apache Mesos handles the launching
> of containers and limiting containers resources so that slaves won’t starve
> each other out for resources while the Jenkins Mesos plugin sits on top of
> Mesos and starts the slave.jar inside of the container which will then run
> the Jenkins building the BigTop package you want or doing whatever tests you
> need in the isolated environment. It would also not require much changing of
> our current Jenkins setup to get it integrated and build steps for most
> projects would be unchanged. You would just install mesos-master somewhere
> and then install mesos-slave on all of the servers that currently do job
> builds, and Mesos will handle using them all and spinning up the Jenkins
> slaves.
> While the above allows you to do BigTop builds in containers using Jenkins,
> we should also do some more things to make life easier. We can use the
> Jenkins Mesos plugin to run Docker in Docker (dind) so that inside of our
> Jenkins slave containers we can launch a Docker container inside of that to
> build Docker images for deployment. For example, you could have a Jenkins job
> that is kicked off inside of a container that then starts and builds a base
> Docker image that will later build a specific package on another job. This
> would allow our Jenkins setup to build and test our Puppet toolchain code and
> then produce images that would be used by other jobs for builds. This is
> documented and talked a lot about here:
> http://www.ebaytechblog.com/2014/04/04/delivering-ebays-ci-solution-with-apache-mesos-part-i/#.U7R4t3VdVhF
>
> As for matching builds on peoples computers, that gets pretty easy with
> Docker. You can just take the Jenkins Slave docker image and then run jobs
> against it without starting the slave.jar. So instead of docker run
> supervisord which you would do to get the Jenkins slave running, you can do
> docker run make hive-deb or whatever package you wanted. This would make
> peoples environments match our Jenkins environment exactly.
> I have an experimental git repo with some Dockerfiles that are a pretty good
> prototype of this future setup located at
> https://github.com/jeid64/bigtop-dockerfiles jenkins-docker/ has an example
> Ubuntu 12.04 Dockerfile that once built uses the Puppet manifests in
> bigtop_toolchain to setup the build environment. When run by Jenkins, Jenkins
> will start a slave inside of the container and the slave would handle all the
> build steps for that build. That image can also just be used on your desktop
> without needing a Jenkins master or using anything from Jenkins.
> Also in that repo is jenkins-dind/ which is based off of
> https://github.com/ahunnargikar/jenkins-dind multi-docker setup. This will be
> the image for dind image builds in Jenkins for build environments.
> I would love to head up this ticket and work on getting everything setup.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
