I really like the Cascading recipes above, thanks for sharing that ! Also we have *apache bigtop vagrant recipes* which we curate for this kind of thing, which are really useful. You can spin up a 1 or multi node cluster, just by running the startup.sh script.
Which are probably the most configurable and flexible. These are super easy to use, and allow you maximal control over your environment. 1) git clone https://github.com/apache/bigtop 2) cd bigtop-deploy/vm/vagrant/vagrant-puppet 3) Follow the directions in the README to create your hadoop cluster You can look into the provision script to see how you can customize exactly which components come (hbase,mahout,pig,....) installed in your distribution. Feel free to drop a line on the bigtop mailing list if you need any help with getting them up and running. On Sun, Jul 6, 2014 at 12:47 PM, Andre Kelpe <[email protected]> wrote: > We have a multi-vm or single-vm setup with apache hadoop, if you want to > give that a spin: > https://github.com/Cascading/vagrant-cascading-hadoop-cluster > > - André > > > On Sun, Jul 6, 2014 at 9:05 AM, MrAsanjar . <[email protected]> wrote: > >> For my hadoop development and testing I use LXC (linux container) instead >> of VM, mainly due to its light weight resource consumption. As mater of >> fact as I am typing, my ubuntu system is automatically building a 6 nodes >> hadoop cluster on my 16G labtop. >> If you have an Ubuntu system you could install a fully configurable >> Hadoop 2.2.0 single node or multi-node cluster in less then 10 minutes. >> Here what you need to do: >> 1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions : >> https://juju.ubuntu.com/docs/getting-started.html >> 2) there are two types hadoop charms: >> a) Single node for hadoop development : >> https://jujucharms.com/?text=hadoop2-devel >> b) multi-node for testing testing : >> https://jujucharms.com/?text=hadoop >> Let me know if you need more help >> >> >> On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <[email protected]> wrote: >> >>> Note that the CDH link is for Cloudera which only provides Hadoop for >>> Linux. >>> >>> HDP has "pre-built VMs" for both Linux and Windows hosts. >>> >>> You can also search for "HDInsight emulator" which runs on Windows and >>> is based on HDP. >>> >>> Marco >>> >>> On Jul 6, 2014, at 12:38 AM, Gavin Yue <[email protected]> wrote: >>> >>> http://hortonworks.com/products/hortonworks-sandbox/ >>> >>> or >>> >>> CDH5 >>> >>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html >>> >>> >>> >>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> I am a newcomer in using Hadoop, and I read many online tutorial to set >>>> up Hadoop on Window by using virtual machines, but all of them link to old >>>> versions of Hadoop virtual machines. >>>> Could any one help me to find a Hadoop virtual machine, which include a >>>> newer version of hadoop? Or should I do it myself from scratch? >>>> Also, any well explained Hadoop installing tutorial and any other >>>> helpful material are appreciated. >>>> >>>> >>>> Manar, >>>> >>>> >>>> -- >>>> >>>> >>> >> > > > -- > André Kelpe > [email protected] > http://concurrentinc.com > -- jay vyas
