[
https://issues.apache.org/jira/browse/MAPREDUCE-1380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordà Polo updated MAPREDUCE-1380:
----------------------------------
Attachment: MAPREDUCE-1380_1.1.patch
Patch against trunk.
> Adaptive Scheduler
> ------------------
>
> Key: MAPREDUCE-1380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1380
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Jordà Polo
> Priority: Minor
> Attachments: MAPREDUCE-1380_0.1.patch, MAPREDUCE-1380_1.1.patch
>
>
> The Adaptive Scheduler is a pluggable Hadoop scheduler that automatically
> adjusts the amount of used resources depending on the performance of jobs and
> on user-defined high-level business goals.
> Existing Hadoop schedulers are focused on managing large, static clusters in
> which nodes are added or removed manually. On the other hand, the goal of
> this scheduler is to improve the integration of Hadoop and the applications
> that run on top of it with environments that allow a more dynamic
> provisioning of resources.
> The current implementation is quite straightforward. Users specify a deadline
> at job submission time, and the scheduler adjusts the resources to meet that
> deadline (at the moment, the scheduler can be configured to either minimize
> or maximize the amount of resources). If multiple jobs are run
> simultaneously, the scheduler prioritizes them by deadline. Note that the
> current approach to estimate the completion time of jobs is quite simplistic:
> it is based on the time it takes to finish each task, so it works well with
> regular jobs, but there is still room for improvement for unpredictable jobs.
> The idea is to further integrate it with cloud-like and virtual environments
> (such as Amazon EC2, Emotive, etc.) so that if, for instance, a job isn't
> able to meet its deadline, the scheduler automatically requests more
> resources.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
