[
https://issues.apache.org/jira/browse/YARN-5959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742711#comment-15742711
]
Arun Suresh edited comment on YARN-5959 at 12/12/16 6:33 PM:
-------------------------------------------------------------
Thanks for capturing the salient points from our discussion [~leftnoteasy].
Updating patch based your feedback..
1.
bq. Instead of creating a hard-locality container request, can we wrap the
ResourceRequest to add some additional information to identify it is for
container increase / promotion.
It looks like we already have an {{AbstractResourceRequest}} which both the
{{ResourceRequest}} and {{UpdateContainerRequest}} extends from.
I renamed it to {{SchedulerResourceRequest}} and also included a field
*containerToUpdate* to signify this is an update to an existing container.
2.
bq. if any normal container request with higher priority, it will be allocated
first. However this is discussable: if we give the available resource of a node
to a different normal container, it is possible that the running opportunistic
container will be killed and work will be wasted.
This is IMO still an ordering issue. I have updated the {{SchedulerRequestKey}}
to ensure the behavior you pointed out. Update requests (both promotion as well
as resource updates) will be considered before any normal resourceRequest.
Since the schedulerKey is created using a {{SchedulerResourceRequest}}, which
contains information about the container update and not the old
{{ResourceRequest}}, I do not have to hard code a negative allocationRequestId
as I had done in the previous patch.
3.
{quote}
* Track pending container changes requests.
* Cancel / update pending container change requests
* Makes backend implementation of container increase / promotion to be same.
{quote}
I feel all these cases can be taken care of in the new patch by ensuring that
the {{SchedulerResourceRequest}} and {{SchedulerRequestKey}} understands that
it is pertaining to a container update.
Once you agree on this new design, I can go ahead (I think we can do that in a
separate JIRA) and create a patch that unifies the backend for container
resource increase and promotion.
was (Author: asuresh):
Thanks for capturing the salient points from our discussion [~leftnoteasy].
Updating patch based your feedback..
1.
bq. Instead of creating a hard-locality container request, can we wrap the
ResourceRequest to add some additional information to identify it is for
container increase / promotion.
It looks like we already have an {{AbstractResourceRequest}} which both the
{{ResourceRequest}} and {{UpdateContainerRequest}} extends from.
I renamed it to {{SchedulerResourceRequest}} and also included a field
*containerToUpdate* to signify this is an update to an existing container.
2.
bq. if any normal container request with higher priority, it will be allocated
first. However this is discussable: if we give the available resource of a node
to a different normal container, it is possible that the running opportunistic
container will be killed and work will be wasted.
This is IMO still an ordering issue. I have updated the {{SchedulerRequestKey}}
to ensure the behavior you pointed out. Update requests (both promotion as well
as resource updates) will be considered before any normal resourceRequest
3.
{quote}
* Track pending container changes requests.
* Cancel / update pending container change requests
* Makes backend implementation of container increase / promotion to be same.
{quote}
I feel all these cases can be taken care of in the new patch by ensuring that
the {{SchedulerResourceRequest}} and {{SchedulerRequestKey}} understands that
it is pertaining to a container update.
Once you agree on this new design, I can go ahead an create a patch that
unifies the backed for container resource increase and promotion.
> RM changes to support change of container ExecutionType
> -------------------------------------------------------
>
> Key: YARN-5959
> URL: https://issues.apache.org/jira/browse/YARN-5959
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Arun Suresh
> Assignee: Arun Suresh
> Attachments: YARN-5959.combined.001.patch, YARN-5959.wip.002.patch,
> YARN-5959.wip.003.patch, YARN-5959.wip.patch
>
>
> RM side changes to allow an AM to ask for change of ExecutionType.
> Currently, there are two cases:
> # *Promotion* : OPPORTUNISTIC to GUARANTEED.
> # *Demotion* : GUARANTEED to OPPORTUNISTIC.
> This is similar in YARN-1197 which allows for change in Container resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
