[
https://issues.apache.org/jira/browse/YARN-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13829377#comment-13829377
]
Carlo Curino commented on YARN-1434:
------------------------------------
This has been observed while modifying the mapreduce AM behavior for other
reasons. If the AM aggressively returns containers, it seems to be able to
create the illusion to be under-capacity while wasting resources for everyone.
A second job running in a separate queue (which was supposed to receive 50% of
the cluster resources) was starved (only getting about 30% of the resources).
This should be confirmed independently as the environment we observed this in
had too much going on (i.e., this might be a false positive).
If confirmed, this might be quite bad, as a single malevolent AM could affect
the cluster utilization possibly by a lot.
[~sandyr], [~acmurthy] thoughts?
> Single Job can affect fairshare of others
> -----------------------------------------
>
> Key: YARN-1434
> URL: https://issues.apache.org/jira/browse/YARN-1434
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Reporter: Carlo Curino
> Priority: Minor
>
> A job receiving containers and deciding not to use them and yielding them
> back in the next heartbeat could significantly affect the amount of resources
> given to other jobs.
> This is because by yielding containers back the job appears always to be
> under-capacity (more than others) so it is picked to be the next to receive
> containers.
> Observed by Robert Grandl, to be independently confirmed.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
