tillrohrmann commented on a change in pull request #7356:
[FLINK-10868][flink-yarn] Enforce maximum TMs failure rate in ResourceManagers
URL: https://github.com/apache/flink/pull/7356#discussion_r258503759
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/ResourceManager.java
##########
@@ -425,6 +462,13 @@ public void disconnectJobManager(final JobID jobId, final
Exception cause) {
slotRequest.getJobId(),
slotRequest.getAllocationId());
+ if (failureRater.exceedMaximumFailureRate()) {
+ return
FutureUtils.completedExceptionally(new
MaximumFailedTaskManagerExceedingException(
+ new
RuntimeException(String.format("Maximum number of failed container %d in
interval %s "
+ + " is detected in
Resource Manager.", failureRater.getCurrentFailureRate(),
+
failureRater.getFailureInterval().toString()))));
+ }
Review comment:
I think this is the wrong place to check this. We should check the
`failureRater.exceedMaximumFailureRater()` in a method from which we call
`startNewWorker`. E.g. `tryStartNewWorker`.
```
protected Collection<ResourceProfile> tryStartNewWorker(ResourceProfile
resourceProfile) {
if (failureRater.exceedMaximumFailureRate()) {
return Collections.emptyList();
}
return startNewWorker(resourceProfile);
}
protected void startNewWorkerIfNeeded(ResourceProfile resourceProfile, int
pendingSlots) {
int currentPendingSlots = pendingSlots;
while (currentPendingSlots < getNumberRequiredTaskManagerSlots()) {
final Collection<ResourceProfile> slots =
tryStartNewWorker(resourceProfile);
if (slots.isEmpty()) {
break;
}
currentPendingSlots += slots.size();
}
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
