On Sun, Dec 27, 2015 at 03:20PM, Evans Ye wrote: > Hey, sorry for disappearing for a while, I was exhausted by a project in > trend micro...
Yeah, we all have $dayjobs, I know the feeling ;( > The reason we don't have ubuntu is just because it hasn't been implemented > yet. > So you're doing the right thing. thanks! Ah, cool. I have committed a couple of fixes to the source code and added a CI job to push the deploy images into our docker repo. We can get rid of this later, when anything better be ready. > As one of the developer of the provisioner feature. > My personal reason for not keep implementing this is because I plan to drop > vagrant and embrace docker's native solutions(machine, compose, swarm) for > our docker provisioner. > In that way, we gain better native support, cleaner code, ability to impl > multi-host cluster deployment feature, etc. > And the most important thing is that we don't need to prepare a docker > image for deployment anymore. > This drastically increase the UX of docker provisioner. User can drop any > docker image as base and deploy Bigtop Hadoop stack on top of it. > As of now I only have some initial work for this in private branch. ;) I agree with the sentiment. In fact, my first urge was to try to implement the automatic cluster deployment and testing for CI with docker-compose. However, after spending a couple of hours ago today just trying to unsuccessfully install the environment, led me to believe that this is a very immature stuff, with heavy and pretty hairy python-to-C implementation. And basically isn't ready for prime-time. They might be able to finish it over the course of the next few months, but in the meanwhile we can just stick to the vagrant and move on, if and when compose and swarm are in a better shape. Does it make sense? Whenever you feel like to share your implementation - I would be happy to help moving it forward. Cheers, Cos > 2015-12-27 11:14 GMT+08:00 Konstantin Boudnik <[email protected]>: > > > On Sat, Dec 26, 2015 at 09:59PM, Jay Vyas wrote: > > > Deploy images? I thought we killed all the deploy images entirely since > > (1) > > > boxgrinder died off, and nobody had the packer expertise to redo > > everything > > > and (2) the vagrant deployer worked effectively for anyone wanting to do > > a > > > quick test. > > > > I am talking about bigtop/deploy:debian-8 and bigtop/deploy:centos-6 docker > > images. > > > > Cos > > > > > > > On Dec 26, 2015, at 7:21 PM, Konstantin Boudnik <[email protected]> > > wrote: > > > > > > > > On a slightly separate note (may be Jay knows the answer): I noticed > > that we > > > > only have centos-6 and debian-8 deploy images. Is there any technical > > reason > > > > for not making them (and limiting our ability to do automatic testing) > > or it > > > > just never been implemented? > > > > > > > > Thanks > > > > Cos > > > > > > > >> On Fri, Dec 25, 2015 at 09:09PM, Konstantin Boudnik wrote: > > > >> Moving a bit of the topics of pure polishing of rough edges, I wanted > > to bring > > > >> up the idea of automatically producing the package repos of the > > nightly > > > >> builds. With a few lines of shell we can do it as demonstrated in > > > >> > > > >> https://github.com/c0s/dumb-ass-scripts/blob/master/gen-apt.sh > > > >> and > > > >> https://github.com/c0s/dumb-ass-scripts/blob/master/gen-yum.sh > > > >> > > > >> Considering that we are dealing here with a matrix of OSes in this > > case, it > > > >> won't be perhaps ideal to roll this functionality into the main > > build. Keeping > > > >> it as an aux helper seems like a better idea. > > > >> There's of course disk-consumption issue, but we might be able to > > solve it. > > > >> > > > >> Thoughts? > > > >> Cos > > > >> > > > >>> On Wed, Dec 23, 2015 at 12:34PM, Konstantin Boudnik wrote: > > > >>> Guys, > > > >>> > > > >>> I've been trying to replicate our CI elsewhere and here's a couple of > > > >>> observations and proposed fixes that might do such things easier in > > the > > > >>> future. > > > >>> > > > >>> 1. Running build as root inside of the docker container. > > > >>> > > > >>> This seems like a real issue, especially considering that we have > > always > > > >>> advocated to stay away from such practice. Unfortunately, adding > > > >>> -u jenkins:jenkins > > > >>> to docker run snags on a couple of points > > > >>> > > > >>> 2. Shared Gradle directory shouldn't belong to root, or at least > > should be > > > >>> writable for everyone. > > > >>> > > > >>> This is covered in BIGTOP-2171 (appreciate the review) and has > > caused user > > > >>> confusions like BIGTOP-2184 > > > >>> > > > >>> 3. One perhaps last issue here is the discrepancy between the user > > ids, where > > > >>> jenkins on centos and ubuntu have different UID (BIGTOP-2187) > > > >>> > > > >>> I think with these three in place, we should be able to start using > > > >>> un-privileged user for the builds and also for the cluster testing. > > > >>> > > > >>> Thoughts? > > > >>> Cos > > > > > > > > > >
signature.asc
Description: Digital signature
