How much RAM do you have? A good rule of thumb is to use 1-1.5G for maps and 2G per reduce (vmem). Ensure your OS has at least 2G of memory.
Thus, with 24G and dual quad cores you should be at 8-10m/2r. Scale up if you have more memory. Also, ensure you turn on memory monitoring so that a rouge task won't overrun a node and cause your TT/DN to swap... that would very bad for the cluster. hth, Arun Sent from my iPhone On Nov 13, 2011, at 6:57 PM, Tom Hall <thattommyh...@gmail.com> wrote: > Hi All, > > We have been using dual socket quad core machines for a while and have > been running with 8 mappers 2 reducers. The rule of thumb we heard was > slightly oversubscribe the number of cores but at Hadoop World several > people said other things. > Some new machines we are moving to have 2 socket 6 core machines with > hyperthreading (24 'cores' if you count each hyperthreaded one as 2) > > What are people doing for M/R ratios and number of M+R per machine at > the moment? > > Thanks, > Tom
