YARN doesn't know about maps or reduces... With CapacityScheduler, this situation you described gets averted since resources are not 'peanut buttered' among too many jobs or users, and within each job reduces resource-requests have higher priority anyway.
hth, Arun On Feb 25, 2013, at 1:03 PM, lohit wrote: > Hi Devs, > > With YARN, is there a way to limit max number of map tasks per cluster. For > example, if we have thousands of jobs running we want to make sure there > are room for reducers and not all mappers from all job occupy resources. > Is there some config people can use to restrict or divide map/reduce. At > first I thought it could be scheduler config, but I could not find any > information about this. > > -- > Have a Nice Day! > Lohit -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
