[
https://issues.apache.org/jira/browse/FLINK-7870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235846#comment-16235846
]
ASF GitHub Bot commented on FLINK-7870:
---------------------------------------
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/4887#discussion_r148553179
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/ResourceManager.java
---
@@ -399,6 +399,26 @@ public void disconnectJobManager(final JobID jobId,
final Exception cause) {
}
@Override
+ public void cancelSlotRequest(JobID jobID, JobMasterId jobMasterId,
AllocationID allocationID) {
+
+ // As the slot allocations are async, it can not avoid all
redundent slots, but should best effort.
+ JobManagerRegistration jobManagerRegistration =
jobManagerRegistrations.get(jobID);
+
+ if (null != jobManagerRegistration) {
+ if (Objects.equals(jobMasterId,
jobManagerRegistration.getJobMasterId())) {
+ log.info("Cancel slot request for job {} with
allocation id {}.",
+ jobID, allocationID);
+
+ slotManager.unregisterSlotRequest(allocationID);
+ } else {
+ log.info("Job manager {} is not the leader of
job {}.", jobMasterId, jobID);
+ }
+ } else {
+ log.warn("Could not find registered job manager for job
{}.", jobID);
+ }
--- End diff --
This could be simplified to
`slotManager.unregisterSlotRequest(allocationId)` if we change
`cancelSlotRequest(JobID, JobMasterId, AllocationID)` to
`cancelSlotRequest(AllocationID)`.
> SlotPool should cancel the slot request to RM if not need any more.
> -------------------------------------------------------------------
>
> Key: FLINK-7870
> URL: https://issues.apache.org/jira/browse/FLINK-7870
> Project: Flink
> Issue Type: Bug
> Components: Cluster Management
> Reporter: shuai.xu
> Assignee: shuai.xu
> Priority: Major
> Labels: flip-6
>
> 1. SlotPool will request slot to rm if its slots are not enough.
> 2. If a slot request is not fulfilled in a certain time, SlotPool will treat
> the request as timeout and send a new slot request by triggering a failover
> in JobMaster, the previous request is not needed any more, but rm does not
> know it.
> 3. This may cause the rm request much more resource than the job really need.
> For example:
> 1. A job need 100 slots. RM request 100 container to YARN.
> 2. But YARN is busy now, it has no resource for the job.
> 3. The job failover as the resource request not fulfilled in time.
> 4. It ask 100 slots again, now RM request 200 container to YARN.
> 5. If failover server time, the containers request will become more and more.
> 6. Now YARN has resource, it will find that the job may need thousands of
> containers. This is a waste of resources.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
