Pavan, A very important factor is how much CPU and how many spindles you have...
Your proposal for memory (44G in all) seems reasonable. However, if you have 12 spindles and sufficient CPU I'd do something like 10 or 12 maps of 2G each and 6 reduces with 3G/4G each depending on how you want to slice/dice your slots. Arun On Aug 10, 2012, at 1:24 PM, Pavan Kulkarni wrote: > Hi, > > I was trying to optimize Hadoop-1.0.2 performance by setting > *mapred.tasktracker.map.tasks.maximum > ,**mapred.tasktracker.reduce.tasks.maximum* > such that the entire memory is utilized. The tuning of this parameter is > given as (CPUS > 2) ? (CPUS * 0.50): 1 for reduce and (CPUS > 2) ? (CPUS * > 0.75): 1 for map. > I didn't quite get how they made this suggestion ? Isn't the setting > dependent on main memory available? > For example I had 48GB of memory and I split the parameters as 32 for > mappers and 12 for reducers and remaining 4 for OS and other processes. > Please correct me if my assumption is wrong.Also suggest a way to get the > optimal performance by setting these parameters. Thanks. > > -- > > --With Regards > Pavan Kulkarni -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
