[
https://issues.apache.org/jira/browse/BIGTOP-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054340#comment-14054340
]
Julien Eid commented on BIGTOP-1323:
------------------------------------
Hi Roman,
I’m really glad to see this ticket come up, I’ve been doing some work on
getting Jenkins slave containerized builds going as well as building Bigtop in
containers so I was very excited to see this! I totally agree with everything
you said about our current Jenkins setup and I’ve seen the hell that can come
from having a matrix of build platforms and slaves that are all configured a
different way and are very brittle. I have a bunch of recommendations on how we
can fix this issue and get our build process cleaned up and working
automagically without human intervention while making our build environment on
Jenkins match the build environment on people’s personal computers.
I have made a separate JIRA issue to talk about how to integrate our Jenkins
setup with Docker containers. I separated the integration of the Jenkins side
of things from this ticket as it appeared to me to be a separate but related
issue to having the ability to have builds be done in containers. The link can
be found here: https://issues.apache.org/jira/browse/BIGTOP-1368
As for just doing builds in containers, I have done a fair chunk of work on
getting BigTop builds happening in Docker containers. I have an experimental
git repo with some Dockerfiles that are a pretty good prototype of doing Bigtop
builds in containers as well having some prototype Dockerfiles for my seperate
Jenkins ticket. It’s located at https://github.com/jeid64/bigtop-dockerfiles
The build/ folder has a working Ubuntu 12.04 Dockerfile that will use the
Puppet modules for the toolchain to configure the container to do builds later
on. This Dockerfile works as far as I can see, I’ve built both HBase and Hive
in a container. We can use this Docker image to build on Jenkins as well as on
dev’s own computers. I’ve built the image and pushed it to the public Docker
hub. You can use my Docker image located here with the following command:
docker run -i -t jeid64/build-precise bash -c 'source /etc/profile.d/bigtop.sh;
cd /bigtop ; make hbase-deb;'
It currently does not put the artifacts anywhere, but that can be handled
easily by a mount volume that gets the resultant packages and is accessible by
the host.
That Dockerfile for 12.04 can be quickly copied and modified for different
versions of Ubuntu. I can make Dockerfiles for our other platforms easily if
they are configurable from Puppet for the toolchain. I saw with Bigtop-1359
there was some work to add more platforms and I think that’s the best way to
have the ability to easily make containers with the toolchain. I’ve already
created a Centos 6.4 image and will work on supporting all of our platforms.
I would love to be assigned to this ticket along with anyone else that wants to
work on it and get everything setup.
> provide an option to containerize Bigtop package build
> -------------------------------------------------------
>
> Key: BIGTOP-1323
> URL: https://issues.apache.org/jira/browse/BIGTOP-1323
> Project: Bigtop
> Issue Type: Improvement
> Components: Build
> Affects Versions: 0.8.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.8.0
>
>
> Looking at the state of our Jenkins made me scream "I'm mad as Hell and I'm
> not going to take this anymore". The basic problem is simple: our build
> slaves are not actually managed by Puppet (even though we do have Puppet code
> to do that) and they seem to bitrot quite a bit (e.g. git clone from ASF
> repos on SLES and CentOS5 is totally borked).
> There's another, somewhat subtler, problem: currently we have to keep as many
> slaves as we've got platforms to support. This leads to a pretty poor
> utilization and if a slave goes down requires manual intervention.
> What I'd like to introduce in this JIRA is a notion of decoupling OS
> environment dependency from the base OS. The technology I'm proposing here is
> Docker (which really is just a prettier UI on top of Linux Containers). IOW,
> regardless of what OS you'd be running on your host -- as long as you've got
> docker you can have an exact replica of a target os running in a container.
> Here's my plan for simplifying the management aspect of what's going on.
> Note, that this is a complimentary plan. Whatever is happening (or not
> happening!) today on our Jenkins can remain there for as long as we need it.
> # make our build slaves be 100% uniform and fungible instances of CoreOS
> VMs https://coreos.com/ The benefit of CoreOS is pretty much 0 administration
> cost, quick boot up time, full integration with Docker (see bellow) and full
> integration with things like VirtualBox, etc (so that you can replicate
> exactly the kind of process that we will setup on the Jenkins).
> # make the actual builds happen in a container using CoreOS's integration
> with Docker. IOW, instead of running: make foo on the slave the actual
> command will look like: docker run -i -t -v `pwd`:/workspace:rw BUILD-ENV
> bash -c 'cd /workspace ; make foo'
> # add jobs to the jenkins that would maintain images of BUILD-ENV
> containers. Each image will be a base-line OS (RHEL, Ubuntu, etc) plus all
> the build dependencies of a given source package (but nothing else!).
> All of these 3 steps are pretty easy to automate and they do have an added
> benefit of being 100% replicatable on your workstation. IOW, as long as you
> have docker running on your host OS you can build packages for any OS with a
> simple command.
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
