The missing note in the document is that the cluster admin need to setup some general environment settings such as the password-less ssh. Definitely it would be great if we can provide some suggestions or ways to do that, but I think somehow you still need to setup that during machine provisioning(installing OS, setting up hostname, ip, password, ssh keys), right?
If you have bunch of machines ready for the deployment, here's the step to setup a bigtop cluster: 1) install puppet on each node 2) install puppet modules on each node puppet apply --modulepath= -e "include bigtop_toolchain::puppet-modules" 2) propagate puppet recipes and configurations <https://github.com/apache/bigtop/tree/master/bigtop-deploy/puppet>on each node 3) run puppet apply on each node 4) done! The puppet and module installation can be done through cluster admin tool such as pdsh, which allows you to run same command across the whole cluster in parallel mode. The puppet recipes or bigtop repo can be put in a share storage such as NFS so that it is available for all nodes. After all of this are set, you can then run puppet apply on each node through pdsh as well. This should give you an up and running bigtop cluster. If you're not quite sure how to do that, you can also refer to bigtop's vagrant provisioner <https://github.com/apache/bigtop/tree/master/bigtop-deploy/vm/vagrant-puppet-vm>, which demonstrate the whole process to setup a bigtop cluster by a click of button. Of course, you can ask questions in mailing list when you encounter any issue. We're happy to help. :) 2015-06-06 6:30 GMT+08:00 Demai Ni <[email protected]>: > hi, folks, > > I am looking at BigTop as noncommercial hadoop distribution package, as an > alternative of CDH, HDP. And this is my first time to try out bigTop. > > My expectation was: > 1) setup my cluster with passwordless ssh > 2) download the necessary bigtop components on one of the nodes. > 3) run an installation/configure tool from the node on 2) > 4) BigTop will (through puppet) install and configure the whole cluster. > > the point is that no need of a lot of manual work on every node. > > But I am following instruction for 0.8.0 and 0.7.0, like > > https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+BigTop+0.7.0+hadoop+on+CentOS+with+puppet > > there are quite a few steps, (almost all the steps in above link) are > required to be performed on each node? > > so it looks like the human work is significant higher between small(3 > node) and large(100+) cluster. Is that right? > > Thanks. > > Demai > >
