Hi Mike, Against most people's better judgment, I am running a hadoop/hbase cluster on previously unused desktops that were sitting in my company's basement. Not to be self promoting, but I documented some of the tweaks I did to make it run with very good stability on my website at http://www.travishegner.com/2009/11/hbase-020-low-memory-performance-tuning.html. The document is a little dated, but the basic concept was to cut everything in half. I like to think of it as a mini-cluster.
I set it up mostly for learning about parallel computing, but now we use it in pseudo-production to house historical server logs for distributed grep, and (low volume) website analytics. I've been collecting various data from our network for about 6-8 months and have only accumulated 21GB worth... Even a small amount has proven to be quite useful for us. I run my master as a virtual machine to have some sort of redundancy/protection for it. Mini-Cluster Specs: Hadoop: 0.20.3 HBase: 0.20.5 Raw DFS space: ~1 TB Master: ESXi VM 2 Processors 2GB RAM 20 GB Hard disk NameNode, JobTracker, HMaster, Zookeeper Workers (5 total): Gateway E4100 Pentium 4 HT 3.2Ghz 2GB RAM 250 GB Hard disk DataNode, TaskTracker, RegionServer It's not exactly a top dollar cluster, but it suites our needs pretty well, and it's a great learning experience for me. Thanks, Travis Hegner http://www.travishegner.com/ -----Original Message----- From: [email protected] [mailto:[email protected]] Sent: Tuesday, October 26, 2010 6:01 PM To: [email protected] Subject: Contributing to hbase but test with less hardware Dear HBase devs, I am reading the HBase sources and have also read the http://wiki.apache.org/hadoop/Hbase/HowToContribute wiki page including the jira "noob" label suggestion. If I do not have 4 or 5 machines at home with sufficient RAM to test changes on a meaningful HBase cluster, what are my other alternatives? Apart from companies with large deployments (su, cloudera, y!) and where devs can remotely test their changes, is there a free/cheap cluster for the less fortunate others who own laptops with 2GB RAM? Thanks for your help. Mike The information contained in this communication is confidential and is intended only for the use of the named recipient. Unauthorized use, disclosure, or copying is strictly prohibited and may be unlawful. If you have received this communication in error, you should know that you are bound to confidentiality, and should please immediately notify the sender or our IT Department at 866.459.4599.
