Hi Todd,

Thanks for the reply. I figured out that *userMaxJobsDefault*** was set to 1. I have another query regarding the same. What will happen if I remove *userMaxJobsDefault *property? What is the default value? Would setting a value higher than 1 for a particular user leads other users' jobs to stall till these jobs get over? If so, is there a way where we can set that, a user can take at max some percentage of total idle mappers existing at that time? And, if the threshold exceeds, we can let users to run only some defaults number of jobs at a time? This way, we can avoid stalling other users' jobs and also efficiently utilize the cluster. Kindly clarify.

Thanks
Pallavi


Todd Lipcon wrote:
Hi Pallavi,

This doesn't sound right. Can you visit http://jobtracker:50030/scheduler?advanced and maybe send a screenshot? And also upload the allocations.xml file you're using?

It sounds like you've managed to set either userMaxJobsDefault or maxRunningJobs for that user to 1.

-Todd

On Thu, Jan 14, 2010 at 9:05 PM, Pallavi Palleti <pallavi.pall...@corp.aol.com <mailto:pallavi.pall...@corp.aol.com>> wrote:

    Hi all,

    I am experimenting with fair scheduler in a cluster of 10
    machines. The users are given default values("0") for minMaps and
    minReduces in fair scheduler parameters. When I tried to run two
    jobs using the same username, the fair scheduler is giving 100%
    fair share to first job(needs 2 mappers) and the second
    job(needs10 mappers) is in waiting mode though the cluster is
    totally idle. Allowing these jobs to run simultaneously would take
    only 10% of total available mappers. However, the second job is
    not allowed to run till the first job is over. It would be great
    if some one can suggest some parameter tuning which can allow
    efficient utilization of cluster. Efficient I mean, allowing jobs
    to run when the cluster is idle rather letting them in waiting
    mode. I am not sure whether setting "minMaps, minReduces" for each
    user would resolve the issue. Kindly clarify.

    Thanks
    Pallavi


Reply via email to