The number of parallel tasks that are run depends on the amount of memory and vcores on your machines and the amount of memory and vcores required by your mappers and reducers. The amount of memory can be set via yarn.nodemanager.resource.memory-mb(the default is 8G). The amount of vcores can be set via yarn.nodemanager.resource.cpu-vcores(the default is 8 vcores).
-Varun From: sandeep das <[email protected]> Reply-To: <[email protected]> Date: Monday, November 2, 2015 at 3:56 PM To: <[email protected]> Subject: Max Parallel task executors Hi Team, I've a cloudera cluster of 4 nodes. Whenever i submit a job my only 31 parallel tasks are executed whereas my machines have more CPU available but still YARN/AM does not create more task. Is there any configuration which I can change to start more MAP/REDUCER task in parallel? Each machine in my cluster has 24 CPUs. Regards, Sandeep
