I think it is a great idea! And, indeed, it's just a very small step - make it user-friendly ;)
I believe what you're proposing below is a much better way to achieve what I've been trying by wrapping Puppet and content distribution into Gradle (you referred to it 'our own deployer' below). That was a bad idea, in retrospect. And for sure - we need the functionality of this kind: we need something that let anyone to deploy a cluster to a pre-provisioned nodes quickly without additional troubles! I am also a big fan of getting rid of lengthy and at times cryptic pages explaining the installation process. Now, the solution you have in mind will be helping with a typical cluster topology like we are usually setting up with head_node, and other flat things of this nature, right? In other words - a simple, "traditional" cluster layout? If so, may I ask a few questions: - how that topology will be specified? - how it is going to be compatible with what we have right now? Not like I try to preserve what we have right now, as there's not much to preserve. We don't really have a way to desribe a cluster's topology at the moment anyway. - in case we need to describe more complex topology - how would be done? I also presume, that the proposed solution will take advantage of existing Puppet recipes, right? Thanks in advance for your answers! Cos On Tue, Feb 24, 2015 at 09:32AM, jay vyas wrote: > Hi folks. > > This last year we've made a lot of progress, > - we can now easily build bigtop, > - and we can test a fresh build in output/ it in vagrant boxes locally > > Whats next? > > In my mind, there is only one last step, to make bigtop a consumer facing > product: We need to make it so you can simply provision a cluster in the > cloud. > As you may recall, David (student at RPI, intern at analytics company last > summer), asked how to do this, and spent about three weeks on it. Recently > Ata Turk, working on the Massachusets Open Cloud @ Boston University, and > former Yahoo researchers, also asked me the same thing. > > Alternative? > > Well we currently these docs about how to setup 0.7.0, 0.8.0, and so on, > which mostly are an untested, human readable version of our vagrant > recipes. yikes ! > > How to implement a ssh cloud bigtop installation ? > > We could roll our own deployer, but in doing so, we would have to make > semantics for: > > - possible need for machiene reboots > - syncing local folders to remote folders > - installing (and reinstalling) > > luckily, our buddy Vagrant already does this for us (respectively , with > the "reload","synced.folder", and "up" options). > > Additionally, *** I THINK *** we can literally use the exact same vagrant > recipes which we are already using to test --- so we will have a great user > experience, and really easy to reproduce bugs and test deployments. > > > I';l hash out details in BIGTOP-1702 , any thoughts, questions, suggestions > on the implementation or feature? I think it will be easy, but also, it > will be one of the most powerful things we add to bigtop, basically > allowing users to easily use the system... maybe even too easy :) > > -- > jay vyas
