Thanks Arun, On Mon, Nov 14, 2011 at 4:34 AM, Arun Murthy <a...@hortonworks.com> wrote: > How much RAM do you have? > > A good rule of thumb is to use 1-1.5G for maps and 2G per reduce > (vmem). Ensure your OS has at least 2G of memory. > > Thus, with 24G and dual quad cores you should be at 8-10m/2r. Scale up > if you have more memory.
Would you say RAM was the main factor? We currently have 1G heap per mapper. We had heard multiples of 1 disk / 2 core / 4G were good with slightly more slots for (mappers + reducers) than cores. Would you agree? Can you speak to how we should use hyperthreading, can I treat them as separate cores? (I know in virtualisation that the recommendation is to disable it but for some other workloads you get 2x performance improvement) Tom
