[
https://issues.apache.org/jira/browse/MAPREDUCE-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13659278#comment-13659278
]
Sergey Tryuber commented on MAPREDUCE-3859:
-------------------------------------------
Mike, that's great! So I think this task can be closed, unless someone from
Cloudera (their MR1 in CDH4 is still be affected) wants to take care about this
issue and port the fix to old Capacity Scheduler into their sources.
For the others who faces this issue, below is a brief step-by-step instruction
for CDH4.1.2:
* Download sources from https://ccp.cloudera.com/display/SUPPORT/CDH+Downloads.
Note: you need hadoop-0.20-mapreduce-0.20.2+1265 tarball.
* Unpack it and go to root directory.
* Apply changes from the first comment and test case from attached patch.
* Also you should add the following lines:
{code}
reactor.repo=https\://repository.cloudera.com/content/repositories/snapshots
version=2.0.0-mr1-cdh4.1.2
{code}
into src/contrib/index/ivy/libraries.properties and
src/contrib/capacity-scheduler/ivy/libraries.properties files.
* Test fixes that were made:
{code}
ant test-contrib
{code}
* Build a jar file:
{code}
cd src/contrib/capacity-scheduler/
ant jar
cd -
{code}
* The result file will be placed at
build/contrib/capacity-scheduler/hadoop-capacity-scheduler-2.0.0-mr1-cdh4.1.2.jar.
* Replace original file with the fixed on a node where JobTracker is started.
Original file is placed in
/usr/lib/hadoop-0.20-mapreduce/contrib/capacity-scheduler/ directory.
> CapacityScheduler incorrectly utilizes extra-resources of queue for
> high-memory jobs
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3859
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3859
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: capacity-sched
> Affects Versions: 1.0.0
> Environment: CDH3u1
> Reporter: Sergey Tryuber
> Assignee: Sergey Tryuber
> Attachments: test-to-fail.patch.txt
>
>
> Imagine, we have a queue A with capacity 10 slots and 20 as extra-capacity,
> jobs which use 3 map slots will never consume more than 9 slots, regardless
> how many free slots on a cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
