tillrohrmann commented on a change in pull request #11353: [FLINK-16438][yarn]
Make YarnResourceManager starts workers using WorkerResourceSpec requested by
SlotManager
URL: https://github.com/apache/flink/pull/11353#discussion_r404160153
##########
File path:
flink-yarn/src/main/java/org/apache/flink/yarn/YarnResourceManager.java
##########
@@ -412,30 +439,32 @@ private void
releaseFailedContainerAndRequestNewContainerIfRequired(ContainerId
final ResourceID resourceId = new
ResourceID(containerId.toString());
// release the failed container
- workerNodeMap.remove(resourceId);
+ YarnWorkerNode yarnWorkerNode =
workerNodeMap.remove(resourceId);
resourceManagerClient.releaseAssignedContainer(containerId);
// and ask for a new one
- requestYarnContainerIfRequired();
+
requestYarnContainerIfRequired(yarnWorkerNode.getContainer().getResource());
}
private void returnExcessContainer(Container excessContainer) {
log.info("Returning excess container {}.",
excessContainer.getId());
resourceManagerClient.releaseAssignedContainer(excessContainer.getId());
}
- private void removeContainerRequest(AMRMClient.ContainerRequest
pendingContainerRequest) {
- numPendingContainerRequests--;
-
- log.info("Removing container request {}. Pending container
requests {}.", pendingContainerRequest, numPendingContainerRequests);
-
+ private void removeContainerRequest(AMRMClient.ContainerRequest
pendingContainerRequest, WorkerResourceSpec workerResourceSpec) {
+ log.info("Removing container request {}.",
pendingContainerRequest);
+ pendingWorkerCounter.decreaseAndGet(workerResourceSpec);
Review comment:
Not saying that we have to implement it right away. I just want to know how
one could fix this as a follow-up task.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
