[
https://issues.apache.org/jira/browse/YARN-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772286#comment-16772286
]
Íñigo Goiri commented on YARN-999:
----------------------------------
I think [^YARN-999.001.patch] is ready for review.
* When the resources were changed using Admin/REST interfaces, the NM didn't
get updated. On the other hand, when we trigger it through the configuration,
it does. I added {{RMNode#isUpdatedCapability()}} to handle this.
* I added the logic for the preemption in
{{AbstractYarnScheduler#killContainersIfOvercommitted()}}. It could be done in
FS or CS but I think this is more general. Maybe we can make it overridable.
* I tweaked the {{TestCapacityScheduler#testResourceOverCommit()}} and at the
end I added a sequence to test the feature. It could technically be split in
smaller pieces.
Thoughts?
> In case of long running tasks, reduce node resource should balloon out
> resource quickly by calling preemption API and suspending running task.
> -----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-999
> URL: https://issues.apache.org/jira/browse/YARN-999
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: graceful, nodemanager, scheduler
> Reporter: Junping Du
> Priority: Major
> Attachments: YARN-291.000.patch, YARN-999.001.patch
>
>
> In current design and implementation, when we decrease resource on node to
> less than resource consumption of current running tasks, tasks can still be
> running until the end. But just no new task get assigned on this node
> (because AvailableResource < 0) until some tasks are finished and
> AvailableResource > 0 again. This is good for most cases but in case of long
> running task, it could be too slow for resource setting to actually work so
> preemption could be hired here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
