[
https://issues.apache.org/jira/browse/HADOOP-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637146#action_12637146
]
Vivek Ratan commented on HADOOP-4211:
-------------------------------------
Yes, part of this behavior is explained by HADOOP-4053. If none of the first
three jobs is removed, and two additional jobs are submitted by two different
users, then we have 5 users in the system, and each user gets 25% of the
resources. When a slot is free, it is given to Job 4 (since Jobs 1, 2, and 3
don't have any tasks to run as they have completed). Slots are given to job4
till that job/user consumes n/4 slots. Then they're given to job5, up until n/4
slots are consumed by job5 too. With the fix for HADOOP-4053, the limits for
job4 and job5 will be n/2, which is right. If either job4 or job5 does not have
enough tasks to run at limit, additional slots are given to jobs that do have a
need, even though they may be running at limit.
I agree with you - you should re-evaluate this behavior once HADOOP-4053 is
fixed. A lot depends on when jobs are marked complete and removed from the
scheduler, as that determines the current user limit.
> Capacity Scheduler does not divide queue resources properly among users, when
> jobs are submitted one after other.
> -----------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4211
> URL: https://issues.apache.org/jira/browse/HADOOP-4211
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Environment: Mapred Cluster capacity with 204 Maps and 204 Reduces.
> User limit =25% and only one queue.
> Reporter: Karam Singh
> Assignee: Hemanth Yamijala
> Priority: Blocker
> Fix For: 0.19.0
>
>
> Capacity Scheduler does not divide queue resources properly among users,
> when job are submitted one after other. E.g. user limit =25. Say User1's job
> is running. Then user2 submits a job. Then user1's job uses 75% and user2's
> job 25%=user limit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
