[
https://issues.apache.org/jira/browse/MAPREDUCE-1380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated MAPREDUCE-1380:
----------------------------------------
Labels: (was: BB2015-05-TBR)
> Adaptive Scheduler
> ------------------
>
> Key: MAPREDUCE-1380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1380
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Affects Versions: 2.4.1
> Reporter: Jordà Polo
> Assignee: Jordà Polo
> Priority: Minor
> Attachments: MAPREDUCE-1380-branch-1.2.patch,
> MAPREDUCE-1380_0.1.patch, MAPREDUCE-1380_1.1.patch, MAPREDUCE-1380_1.1.pdf
>
>
> The Adaptive Scheduler is a pluggable Hadoop scheduler that automatically
> adjusts the amount of used resources depending on the performance of jobs and
> on user-defined high-level business goals.
> Existing Hadoop schedulers are focused on managing large, static clusters in
> which nodes are added or removed manually. On the other hand, the goal of
> this scheduler is to improve the integration of Hadoop and the applications
> that run on top of it with environments that allow a more dynamic
> provisioning of resources.
> The current implementation is quite straightforward. Users specify a deadline
> at job submission time, and the scheduler adjusts the resources to meet that
> deadline (at the moment, the scheduler can be configured to either minimize
> or maximize the amount of resources). If multiple jobs are run
> simultaneously, the scheduler prioritizes them by deadline. Note that the
> current approach to estimate the completion time of jobs is quite simplistic:
> it is based on the time it takes to finish each task, so it works well with
> regular jobs, but there is still room for improvement for unpredictable jobs.
> The idea is to further integrate it with cloud-like and virtual environments
> (such as Amazon EC2, Emotive, etc.) so that if, for instance, a job isn't
> able to meet its deadline, the scheduler automatically requests more
> resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
