Steven Rand created YARN-7655:
---------------------------------
Summary: avoid AM preemption caused by RRs for specific nodes or
racks
Key: YARN-7655
URL: https://issues.apache.org/jira/browse/YARN-7655
Project: Hadoop YARN
Issue Type: Improvement
Components: fairscheduler
Affects Versions: 3.0.0
Reporter: Steven Rand
Assignee: Steven Rand
We frequently see AM preemptions when
{{starvedApp.getStarvedResourceRequests()}} in
{{FSPreemptionThread#identifyContainersToPreempt}} includes one or more RRs
that request containers on a specific node. Since this causes us to only
consider one node to preempt containers on, the really good work that was done
in YARN-5830 doesn't save us from AM preemption. Even though there might be
multiple nodes on which we could preempt enough non-AM containers to satisfy
the app's starvation, we often wind up preempting one or more AM containers on
the single node that we're considering.
A proposed solution is that if we're going to preempt one or more AM containers
for an RR that specifies a node or rack, then we should instead expand the
search space to consider all nodes. That way we take advantage of YARN-5830,
and only preempt AMs if there's no alternative. I've attached a patch with an
initial implementation of this. We've been running it on a few clusters, and
have seen AM preemptions drop from double-digit occurrences on many days to
zero.
Of course, the tradeoff is some loss of locality, since the starved app is less
likely to be allocated resources at the most specific locality level that it
asked for. My opinion is that this tradeoff is worth it, but interested to hear
what others think as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
