make sure you also have a fast switch, since you will be transmitting data across your network and this will come to bite you otherwise
(roughly, you need one core per hadoop-related job, each mapper, task tracker etc; the per-core memory may be too small if you are doing anything memory-intensive. we have 8-core boxes with 50 -- 33 GB RAM and 8 x 1 TB disks on each one; one box however just has 16 GB of RAM and it routinely falls over when we run jobs on it) Miles 2009/4/2 tim robertson <[email protected]>: > Hi all, > > I am not a hardware guy but about to set up a 10 node cluster for some > processing of (mostly) tab files, generating various indexes and > researching HBase, Mahout, pig, hive etc. > > Could someone please sanity check that these specs look sensible? > [I know 4 drives would be better but price is a factor (second hand > not an option, hosting is not either as there is very good bandwidth > provided)] > > Something along the lines of: > > Dell R200 (8GB is max memory) > Quad Core Intel® Xeon® X3360, 2.83GHz, 2x6MB Cache, 1333MHz FSB > 8GB Memory, DDR2, 800MHz (4x2GB Dual Ranked DIMMs) > 2x 500GB 7.200 rpm 3.5-inch SATA Hard Drive > > > Dell R300 (can be expanded to 24GB RAM) > Quad Core Intel® Xeon® X3363, 2.83GHz, 2x6M Cache, 1333MHz FS > 8GB Memory, DDR2, 667MHz (2x4GB Dual Ranked DIMMs) > 2x 500GB 7.200 rpm 3.5-inch SATA Hard Drive > > > If there is a major flaw please can you let me know. > > Thanks, > > Tim > (not a hardware guy ;o) > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
