Evans, thank you very much for the detail instructions.
I understood that the cluster need to be passwordless ssh or someway to provide userid/password of all nodes. So that actions can be propogated to other nodes. I was expecting some tool (pdsh, vagrant provisioner) as you mentioned, would do such job for end-users. But the document didn't mention such tool(or I missed them?), and I got the impression that that steps 1)~3) are required by manual effort. Maybe those tools are common-sense for an experienced cluster administer, so no document is necessary.:-) Anyway, now that with your help, I will try them out. Thanks Demai On Sat, Jun 6, 2015 at 12:03 PM, Evans Ye <[email protected]> wrote: > The missing note in the document is that the cluster admin need to setup > some general environment settings such as > the password-less ssh. Definitely it would be great if we can provide some > suggestions or ways to do that, but I think somehow you still need to setup > that during machine provisioning(installing OS, setting up hostname, > ip, password, ssh keys), right? > > If you have bunch of machines ready for the deployment, here's the step > to setup a bigtop cluster: > > 1) install puppet on each node > 2) install puppet modules on each node > > puppet apply --modulepath= -e "include bigtop_toolchain::puppet-modules" > > 2) propagate puppet recipes and configurations > <https://github.com/apache/bigtop/tree/master/bigtop-deploy/puppet>on > each node > 3) run puppet apply on each node > 4) done! > > The puppet and module installation can be done through cluster admin tool > such as pdsh, which allows you to run same command across the whole > cluster in parallel mode. The puppet recipes or bigtop repo can be put in > a share storage such as NFS so that it is available for all nodes. After > all of this are set, you can then run puppet apply on each node through > pdsh as well. This should give you an up and running bigtop cluster. > > If you're not quite sure how to do that, you can also refer to bigtop's > vagrant > provisioner > <https://github.com/apache/bigtop/tree/master/bigtop-deploy/vm/vagrant-puppet-vm>, > which demonstrate the whole process to setup a bigtop cluster by a click of > button. > > Of course, you can ask questions in mailing list when you encounter any > issue. We're happy to help. :) > > 2015-06-06 6:30 GMT+08:00 Demai Ni <[email protected]>: > >> hi, folks, >> >> I am looking at BigTop as noncommercial hadoop distribution package, as >> an alternative of CDH, HDP. And this is my first time to try out bigTop. >> >> My expectation was: >> 1) setup my cluster with passwordless ssh >> 2) download the necessary bigtop components on one of the nodes. >> 3) run an installation/configure tool from the node on 2) >> 4) BigTop will (through puppet) install and configure the whole cluster. >> >> the point is that no need of a lot of manual work on every node. >> >> But I am following instruction for 0.8.0 and 0.7.0, like >> >> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+BigTop+0.7.0+hadoop+on+CentOS+with+puppet >> >> there are quite a few steps, (almost all the steps in above link) are >> required to be performed on each node? >> >> so it looks like the human work is significant higher between small(3 >> node) and large(100+) cluster. Is that right? >> >> Thanks. >> >> Demai >> >> > >
