Yeah, I am pretty sceptical about master-based setups as well. In my understanding, for proper functioning of such a setup the recipes should be machine-specific or group-specific. While in our case we don't have any up-front information where the recipes will be executed. Hence, our way of using Puppet seems to be pretty well aligned with our use cases.
And getting back to topic: I take it the idea of adding Gradle's end-point for Puppet apply doesn't look too sexy, right? Cos On Sun, Dec 21, 2014 at 11:41PM, Evans Ye wrote: > Regarding to the puppetmaster, I'd like to raise a question that does most > of the company using puppet in this way? > I have experience using puppet master for cluster management and deployment. > But there are troubles I ran into and I think it would be better to switch > to masterless puppet. > When I searched around on net, I found that actually there are many > companies already did the switch. > A typical solution for using masterless puppet is to use version > control(git) with puppet apply. > (though I'm not sure whether there is a better way) > > To increase the Bigtop adoption by improving our puppet stuff, I think > there are some items can be put on for discussion: > > * Journal HA for Namenodes > * find-grained configuration on roles > (some machines run datanode+zookeeper+journalnode, some machines only run > nodemanager) > > However, this is only my personal thoughts. > Would love to hear yours. > > Getting back to the topic, I think the gradle thing can better demonstrate > the usage of bigtop puppet, though some of the ops people might not want > the extra dependency:) > > > 2014-12-20 12:17 GMT+08:00 Konstantin Boudnik <[email protected]>: > > > Thanks for kind words, man - I am glad to see that Gradle took off! > > > > Let me clarify something - sorry for not being clear right away. The > > application of the wrapper isn't to manage LIVE clusters or machines. Also, > > Puppet recipes aren't going away neither. All I had in mind is to provide > > an > > easy wrapper, so that > > > > a) don't bother with copying around deployment script when you need to > > quickly > > get a dev. cluster up and running > > b) one doesn't need to type the long command wrapped in pdsh or otherwise > > > > It might be or might not be a good idea and that's why I wanted to get it > > out > > there for everyone to bash on ;) > > > > Cos > > > > On Fri, Dec 19, 2014 at 09:25PM, jay vyas wrote: > > > Hi cos! I like the goal, but let me pervert your idea a little bit :) > > .... > > > how does this sound ? > > > > > > *** Definetely *** > > > > > > lets keep going w/ gradle for all build/test/deploy stuff. Its great for > > > that ! Full court press to remove maven deps, gradle is wayyyy cleaner > > and > > > definetely is the future trend. the hard work you've done in integrating > > > the whole build to use it is really making bigtop a fun to use packaging > > > system, and actually has made me more able to contribute to it b/c its > > > easier to debug the build now for guys like me that dont know much about > > > make. > > > > > > *** BUT *** > > > > > > although I LOVE the idea of adding a CONTROLLER to the live machines, ... > > > adding it in gradle? very few devops folks will really want to use gradle > > > for that kind of task.... its not the idiom. > > > > > > One of the HUGE selling points of bigtop is that it has puppet recipes, > > > rather than just one of commands/tasks for managing the cluster - the ops > > > community can read and understand those easily. > > > > > > *** SO : PUPPET MASTER .... AN ALTERNATIVE ? *** > > > > > > I think, if we want to manage LIVE machines, then can i suggest a real > > > puppet master implementation? If we go that route, you get all the > > testing > > > for it also, for free, w/ our existing vagrant recipes.... and its in > > line > > > w/ what the devops community is doing, and so it will help drive bigtop > > > adoption. > > > > > > > > > On Fri, Dec 19, 2014 at 8:19 PM, Konstantin Boudnik <[email protected]> > > wrote: > > > > > > > > Guys, > > > > > > > > I've been doing some thinking lately (yes, I know I shouldn't - I've > > heard > > > > this from good friends before, but can't help it) and it makes more and > > > > more > > > > sense to me to wrap more build/deploy functionality into our Gradle > > build. > > > > > > > > The low-hanging fruits I have in mind right now is to provide a > > wrapper for > > > > {{puppet apply bigtop-deploy/puppet/manifests/site.pp}} that will do a > > few > > > > things like: > > > > - copy deployment recipes to all specified hosts, including site.csv > > file > > > > - start puppet apply on all hosts in parallel using gradle ssh plugin > > > > > > > > This, IMO, will provide us with a very easy and transparent way of > > quickly > > > > deploying cluster bits as one works on something, without switching > > out of > > > > the > > > > convenient build environment. Sorta, change-deploy-rinse-and-repeat > > loop. > > > > > > > > The same gradle task can be called, if desired from Jenkins or else. > > > > > > > > Do you think it worth it? Thoughts? > > > > -- > > > > Take care, > > > > Cos > > > > > > > > > > > > > > -- > > > jay vyas > >
signature.asc
Description: Digital signature
