[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643785#comment-16643785 ] Arun Suresh commented on YARN-1013: --- Linking to YARN-8827, since we plan to get the CS to use the user/queue and app utilizations to decide if Opportunistic containers can be doled out to an app. > CS should watch resource utilization of containers and allocate speculative > containers if appropriate > - > > Key: YARN-1013 > URL: https://issues.apache.org/jira/browse/YARN-1013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun C Murthy >Assignee: Arun Suresh >Priority: Major > Attachments: YARN-1013-001.branch-2.patch > > > CS should watch resource utilization of containers (provided by NM in > heartbeat) and allocate speculative containers (at lower OS priority) if > appropriate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619809#comment-16619809 ] Arun Suresh commented on YARN-1013: --- Thanks for taking a quick look [~elgoiri] So, the patch was more of a POC patch (I should have named it as such) I built on top of current branch-2 + some YARN-1011 patches I pulled from that branch - to vet the approach, but yes, I shall clean it up.. and put in a patch for trunk. > CS should watch resource utilization of containers and allocate speculative > containers if appropriate > - > > Key: YARN-1013 > URL: https://issues.apache.org/jira/browse/YARN-1013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun C Murthy >Assignee: Arun Suresh >Priority: Major > Attachments: YARN-1013-001.branch-2.patch > > > CS should watch resource utilization of containers (provided by NM in > heartbeat) and allocate speculative containers (at lower OS priority) if > appropriate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[
https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619771#comment-16619771
]
Íñigo Goiri commented on YARN-1013:
---
Thanks [~asuresh] for [^YARN-1013-001.branch-2.patch].
A couple general questions:
* Can we get a patch for trunk for Yetus to be able to run (branch-2 has
issues)?
* Can you give an overview comparing to the FS approach? I went through the
patch and it is hard to compare as this uses the allocator.
Comments to the patch itself:
* Some of the debug messages seem for development. Should we keep all of them?
* Can you add more comments to {{testContainerOverAllocation()}}? For example,
we setup one node without overallocation and one with it. Why those numbers and
what is the goal?
* Can we add a couple lower level unit tests? Just testing the allocator or the
scheduler?
* There are many space fixes, can we avoid most of them? Specially, pass the
null by default as second parameter to registerNode for TestAMRestart and
TestReservations.
> CS should watch resource utilization of containers and allocate speculative
> containers if appropriate
> -
>
> Key: YARN-1013
> URL: https://issues.apache.org/jira/browse/YARN-1013
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun C Murthy
>Assignee: Arun Suresh
>Priority: Major
> Attachments: YARN-1013-001.branch-2.patch
>
>
> CS should watch resource utilization of containers (provided by NM in
> heartbeat) and allocate speculative containers (at lower OS priority) if
> appropriate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619709#comment-16619709 ] Arun Suresh commented on YARN-1013: --- Attached an initial version of the patch for branch-2. Kindly review.. > CS should watch resource utilization of containers and allocate speculative > containers if appropriate > - > > Key: YARN-1013 > URL: https://issues.apache.org/jira/browse/YARN-1013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun C Murthy >Assignee: Arun Suresh >Priority: Major > Attachments: YARN-1013-001.branch-2.patch > > > CS should watch resource utilization of containers (provided by NM in > heartbeat) and allocate speculative containers (at lower OS priority) if > appropriate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593499#comment-16593499 ] Weiwei Yang commented on YARN-1013: --- Hi [~asuresh], pls go ahead.. I am busy with something else right now so won't be able to come to this one any time soon. Thank you. > CS should watch resource utilization of containers and allocate speculative > containers if appropriate > - > > Key: YARN-1013 > URL: https://issues.apache.org/jira/browse/YARN-1013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun C Murthy >Assignee: Weiwei Yang >Priority: Major > > CS should watch resource utilization of containers (provided by NM in > heartbeat) and allocate speculative containers (at lower OS priority) if > appropriate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593495#comment-16593495 ] Arun Suresh commented on YARN-1013: --- [~cheersyang], If you havn't started with this, wondering if I might take this up... > CS should watch resource utilization of containers and allocate speculative > containers if appropriate > - > > Key: YARN-1013 > URL: https://issues.apache.org/jira/browse/YARN-1013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun C Murthy >Assignee: Weiwei Yang >Priority: Major > > CS should watch resource utilization of containers (provided by NM in > heartbeat) and allocate speculative containers (at lower OS priority) if > appropriate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[
https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526926#comment-16526926
]
Haibo Chen commented on YARN-1013:
--
{quote} where is the enforcement flag?
{quote}
It is per ResourceRequest, included in the ExecutionTypeRequest of a
ResourceRequest. Essentially, a RequestRequest can opt out of oversubscription
by setting its enforcement flag to true. (G, false) requests can start eagerly
as O containers, but there is a possibility that the O containers can sometimes
be preempted if the node is running hot. Applications can decide for themselves
what tasks are critical enough that the risk of starting as O containers and
being preempted is not acceptable. YARN-8240 added control on a queue level,
that is, if a queue opts out of oversubscription, all applications running in
the queue will never get Opportunistic containers for their (G, false)
requests.
{quote}Does this considers resource usages for O container or it is just
consider G container usages?
{quote}
The fair scheduler policy (SchedulingPolicy) is plug-able, so FairScheduler
queues can be sorted with O resource usage of the queue in mind.
> CS should watch resource utilization of containers and allocate speculative
> containers if appropriate
> -
>
> Key: YARN-1013
> URL: https://issues.apache.org/jira/browse/YARN-1013
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun C Murthy
>Assignee: Weiwei Yang
>Priority: Major
>
> CS should watch resource utilization of containers (provided by NM in
> heartbeat) and allocate speculative containers (at lower OS priority) if
> appropriate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524225#comment-16524225 ] Wangda Tan commented on YARN-1013: -- Thanks [~haibochen] for explanations, bq. we are trying to just handle G resource requests with their enforcement flag set to false This is the part I don't quite understand, where is the enforcement flag? Is it per app, per request or globally? bq. but the fair scheduler implementation (YARN-1015) tries to take into account of queue weight ... Does this considers resource usages for O container or it is just consider G container usages? > CS should watch resource utilization of containers and allocate speculative > containers if appropriate > - > > Key: YARN-1013 > URL: https://issues.apache.org/jira/browse/YARN-1013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun C Murthy >Assignee: Weiwei Yang >Priority: Major > > CS should watch resource utilization of containers (provided by NM in > heartbeat) and allocate speculative containers (at lower OS priority) if > appropriate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[
https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524129#comment-16524129
]
Haibo Chen commented on YARN-1013:
--
To add a bit of context to YARN-1015 (or YARN-1011 as a whole), we are trying
to just handle G resource requests with their enforcement flag set to false.
This indicates to the scheduler that it can be satisfied with an Opportunistic
container to start with and the Opportunistic container will later be promoted
to Guaranteed container automatically by the scheduler. If the user does not
want O containers to start with, they set the enforcement flag to true. *O
resource requests are handled by existing OpportunisticAMProcessor.*
{quote}YARN-6794 randomly promotes O container even if user doesn't care about
container execution type.
{quote}
This is not quite right. YARN-6794 promotes O containers in the order they are
allocated. This is to ensure that users can treat YARN-1011 as
eager-scheduling.
Let's say the scheduler receives two requests in order, R1 (G, false) and R2
(G, true) (G is the resourceRequest execution type, and true/false is the the
enforcement flag). If there is no oversubscription, R1 will be satisfied with
a G container at t1 and R2 will be satisfied with a G container at t2 (t1 <
t2). What YARN-1011 does effectively, from the users' point of view, is that R1
will be satisfied with a O container at t0 (t0 < t1), be promoted at time t1,
and R2 will still be satisfied with a G container at t2. You see essentially,
R1 can run as O container from t0 to t1, but it would run as G container right
at the same moment, t1, as if oversubscription never takes place. YARN-6794
has to follow an order to promote O containers so that when there are multiple
(G, false) requests, it still maintains the same order in which the (G, false)
requests would have been satisfied with G containers, if oversubscription had
been turned off.
YARN-8179 is pretty straightforward, and it's probably not as straightforward
to do YARN-1013 with all the other CS features in play, but the fair scheduler
implementation (YARN-1015) tries to take into account of queue weight, resource
usage and application demand, so that Opportunistic resources are allocated the
same way as Guaranteed resources (fair share for Opportunistic resources as
well).
> CS should watch resource utilization of containers and allocate speculative
> containers if appropriate
> -
>
> Key: YARN-1013
> URL: https://issues.apache.org/jira/browse/YARN-1013
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun C Murthy
>Assignee: Weiwei Yang
>Priority: Major
>
> CS should watch resource utilization of containers (provided by NM in
> heartbeat) and allocate speculative containers (at lower OS priority) if
> appropriate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524101#comment-16524101 ] Wangda Tan commented on YARN-1013: -- Just took a very quick look at YARN-1015. IIUC, scheduler allocates O containers when node uses more than guaranteed resource. In my mind, problem of this approach is it cannot guarantee that allocated containers satisfy user's requirement. It doesn't check getExecutionTypeRequest of user's ResourceRequest, and it doesn't consider each app's pending O resource request, and queue's pending O resource request, etc. What if user doesn't want O containers? Similarly, YARN-6794 randomly promotes O container even if user doesn't care about container execution type. The syntax of YARN-8178 is much simpler, application can avoid get O resource request if the resource is not preemptable. I like a proposal from [~curino] that we should add a flag to indicate resource request is Guaranteed and non-preemptable. Once we have that, we can get G container even if queue is preemptable. Considering all CS features (user-limit, node partition, application priority, queue priority) may interact with O containers, I'm not sure how much effort required to cleanly support this in CS. Simply porting YARN-1015 to CS might be oversimplified to me. > CS should watch resource utilization of containers and allocate speculative > containers if appropriate > - > > Key: YARN-1013 > URL: https://issues.apache.org/jira/browse/YARN-1013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun C Murthy >Assignee: Weiwei Yang >Priority: Major > > CS should watch resource utilization of containers (provided by NM in > heartbeat) and allocate speculative containers (at lower OS priority) if > appropriate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523921#comment-16523921 ] Íñigo Goiri commented on YARN-1013: --- Here I would do the same as in YARN-1015 to keep it consistent. We can add a similar approach for YARN-8178 later. > CS should watch resource utilization of containers and allocate speculative > containers if appropriate > - > > Key: YARN-1013 > URL: https://issues.apache.org/jira/browse/YARN-1013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun C Murthy >Assignee: Weiwei Yang >Priority: Major > > CS should watch resource utilization of containers (provided by NM in > heartbeat) and allocate speculative containers (at lower OS priority) if > appropriate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523385#comment-16523385 ] Weiwei Yang commented on YARN-1013: --- Hi [~haibochen], [~elgoiri] I took a look at YARN-1015, it is a similar implementation like what we've done internally. However one thing to double check, what was the conclusion of the discussion the other day between you, [~leftnoteasy] and [~miklos.szeg...@cloudera.com] ? Since YARN-8178 was proposed but a different approach. Do we need to get this one done too and expose both 2 approaches to users? Please let me know, thanks. > CS should watch resource utilization of containers and allocate speculative > containers if appropriate > - > > Key: YARN-1013 > URL: https://issues.apache.org/jira/browse/YARN-1013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun C Murthy >Assignee: Weiwei Yang >Priority: Major > > CS should watch resource utilization of containers (provided by NM in > heartbeat) and allocate speculative containers (at lower OS priority) if > appropriate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519860#comment-16519860 ] Haibo Chen commented on YARN-1013: -- [~cheersyang] FYI, there is YARN-6794 that does container promotion in Fair Scheduler. We have not filed a counter-part Jira for capacity scheduler yet. > CS should watch resource utilization of containers and allocate speculative > containers if appropriate > - > > Key: YARN-1013 > URL: https://issues.apache.org/jira/browse/YARN-1013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun C Murthy >Assignee: Weiwei Yang >Priority: Major > > CS should watch resource utilization of containers (provided by NM in > heartbeat) and allocate speculative containers (at lower OS priority) if > appropriate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate
[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072104#comment-15072104 ] Inigo Goiri commented on YARN-1013: --- I can take this one once YARN-1015 is done. > CS should watch resource utilization of containers and allocate speculative > containers if appropriate > - > > Key: YARN-1013 > URL: https://issues.apache.org/jira/browse/YARN-1013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun C Murthy >Assignee: Arun C Murthy > > CS should watch resource utilization of containers (provided by NM in > heartbeat) and allocate speculative containers (at lower OS priority) if > appropriate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
