[ https://issues.apache.org/jira/browse/YARN-5959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742711#comment-15742711 ]
Arun Suresh edited comment on YARN-5959 at 12/12/16 6:33 PM: ------------------------------------------------------------- Thanks for capturing the salient points from our discussion [~leftnoteasy]. Updating patch based your feedback.. 1. bq. Instead of creating a hard-locality container request, can we wrap the ResourceRequest to add some additional information to identify it is for container increase / promotion. It looks like we already have an {{AbstractResourceRequest}} which both the {{ResourceRequest}} and {{UpdateContainerRequest}} extends from. I renamed it to {{SchedulerResourceRequest}} and also included a field *containerToUpdate* to signify this is an update to an existing container. 2. bq. if any normal container request with higher priority, it will be allocated first. However this is discussable: if we give the available resource of a node to a different normal container, it is possible that the running opportunistic container will be killed and work will be wasted. This is IMO still an ordering issue. I have updated the {{SchedulerRequestKey}} to ensure the behavior you pointed out. Update requests (both promotion as well as resource updates) will be considered before any normal resourceRequest. Since the schedulerKey is created using a {{SchedulerResourceRequest}}, which contains information about the container update and not the old {{ResourceRequest}}, I do not have to hard code a negative allocationRequestId as I had done in the previous patch. 3. {quote} * Track pending container changes requests. * Cancel / update pending container change requests * Makes backend implementation of container increase / promotion to be same. {quote} I feel all these cases can be taken care of in the new patch by ensuring that the {{SchedulerResourceRequest}} and {{SchedulerRequestKey}} understands that it is pertaining to a container update. Once you agree on this new design, I can go ahead (I think we can do that in a separate JIRA) and create a patch that unifies the backend for container resource increase and promotion. was (Author: asuresh): Thanks for capturing the salient points from our discussion [~leftnoteasy]. Updating patch based your feedback.. 1. bq. Instead of creating a hard-locality container request, can we wrap the ResourceRequest to add some additional information to identify it is for container increase / promotion. It looks like we already have an {{AbstractResourceRequest}} which both the {{ResourceRequest}} and {{UpdateContainerRequest}} extends from. I renamed it to {{SchedulerResourceRequest}} and also included a field *containerToUpdate* to signify this is an update to an existing container. 2. bq. if any normal container request with higher priority, it will be allocated first. However this is discussable: if we give the available resource of a node to a different normal container, it is possible that the running opportunistic container will be killed and work will be wasted. This is IMO still an ordering issue. I have updated the {{SchedulerRequestKey}} to ensure the behavior you pointed out. Update requests (both promotion as well as resource updates) will be considered before any normal resourceRequest 3. {quote} * Track pending container changes requests. * Cancel / update pending container change requests * Makes backend implementation of container increase / promotion to be same. {quote} I feel all these cases can be taken care of in the new patch by ensuring that the {{SchedulerResourceRequest}} and {{SchedulerRequestKey}} understands that it is pertaining to a container update. Once you agree on this new design, I can go ahead an create a patch that unifies the backed for container resource increase and promotion. > RM changes to support change of container ExecutionType > ------------------------------------------------------- > > Key: YARN-5959 > URL: https://issues.apache.org/jira/browse/YARN-5959 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Arun Suresh > Assignee: Arun Suresh > Attachments: YARN-5959.combined.001.patch, YARN-5959.wip.002.patch, > YARN-5959.wip.003.patch, YARN-5959.wip.patch > > > RM side changes to allow an AM to ask for change of ExecutionType. > Currently, there are two cases: > # *Promotion* : OPPORTUNISTIC to GUARANTEED. > # *Demotion* : GUARANTEED to OPPORTUNISTIC. > This is similar in YARN-1197 which allows for change in Container resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org