[
https://issues.apache.org/jira/browse/MAPREDUCE-5643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13829820#comment-13829820
]
Hadoop QA commented on MAPREDUCE-5643:
--------------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12615310/README
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4225//console
This message is automatically generated.
> DynamicMR: A Dynamic Slot Utilization Optimization Framework for Hadoop MRv1
> ----------------------------------------------------------------------------
>
> Key: MAPREDUCE-5643
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5643
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/fair-share
> Affects Versions: 1.2.1
> Reporter: tang shanjiang
> Assignee: tang shanjiang
> Labels: performance
> Attachments: DynamicMR-0.1.1-patch, README
>
>
> Hadoop MRv1 uses the slot-based resource model with the static configuration
> of map/reduce slots in advance. Due to the rigid execution order between map
> and reduce tasks in a MapReduce environment and the strict execution
> constrain that map tasks can only run map slots and reduce tasks can only
> reduce slots, slots can be severely under-utilized, which significantly
> degrades the performance.
> In contrast to YARN that gives up the slot-based resource model to maximize
> resource utilization, we keep the slot-based model and propose a dynamic slot
> utilization optimization system called DynamicMR to improve the performance
> of Hadoop by maximizing the slots utilization and improving utilization
> efficiency while guaranteeing the fairness across pools. It consists of three
> levels of scheduling components, namely, Dynamic Hadoop Fair Scheduler
> (DHFS), Dynamic Speculative Task Scheduler (DSTS), and Data Locality
> Maximization Scheduler (DLMS).
> Our tests show that DynamicMR outperforms YARN for MapReduce workloads with
> multiple jobs, especially when the number of jobs is large.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
