[
https://issues.apache.org/jira/browse/MAPREDUCE-944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750775#action_12750775
]
Matei Zaharia commented on MAPREDUCE-944:
-----------------------------------------
I think that rather than passing a LoadManager to the Schedulables and having
them check whether they can run, it's better design to put all the logic in the
LoadManager and ask it whether it's okay to run a particular job. I'd do this
the following way:
* LoadManager contains a method called canLaunchTask(JobInProgress job,
TaskType type, TaskTracker Status tt)
* In JobSchedulable.assignTasks, before looking for a task, the JobSchedulable
first checks whether scheduler.getLoadManager().canLaunchTask(job, type, tt) is
true. If it isn't,
* The default implementation of canLaunchTask always returns true, but in the
memory-aware LoadManager, it can be made to return false.
> Extend FairShare scheduler to fair-share memory usage in the cluster
> --------------------------------------------------------------------
>
> Key: MAPREDUCE-944
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-944
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/fair-share
> Reporter: dhruba borthakur
>
> The FairShare Scheduler has an extensible LoadManager API to regulate
> allocating new tasks on a particular TaskTracker. In similar lines, it would
> be nice if the FairShare Scheduler can have a pluggable policy to regulate
> new tasks from a particular job. This will allow one to skip scheduling tasks
> of a job that is eating a large percentage of memory in the cluster, i.e.
> fair-share of memory resources among jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
