[
https://issues.apache.org/jira/browse/FLINK-10868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16759111#comment-16759111
]
Zhenqiu Huang commented on FLINK-10868:
---------------------------------------
[~till.rohrmann]
Thanks for reviewing the PR. According to your suggestions. I changed as
# As the feature is generic for both Yarn and Mesos, add only the maximum
failure rate config option in resource manager config.
# Put failure rate related logic into the TimestampBasedFailureRater which
implements the FailureRater interface. As I don't want to mix two changes with
different purpose in the same PR, we can make other code
FailureRateRestartStrategy use it in another small PR.
# For the failure rate test for RM, I tried to do in ResourceManagerTest. I
found it is hard to mimic the behavior of registerSlotRequest without mocking
lots components. And I also have to setup Test RM exactly like what
YarnResourceManagerTest is doing. Thus, I still put the test cases separately
in YarnResourceManagerTest and MesosResourceManagerTest.
> Flink's JobCluster ResourceManager doesn't use maximum-failed-containers as
> limit of resource acquirement
> ---------------------------------------------------------------------------------------------------------
>
> Key: FLINK-10868
> URL: https://issues.apache.org/jira/browse/FLINK-10868
> Project: Flink
> Issue Type: Bug
> Components: Mesos, YARN
> Affects Versions: 1.6.2, 1.7.0
> Reporter: Zhenqiu Huang
> Assignee: Zhenqiu Huang
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, YarnResourceManager does use yarn.maximum-failed-containers as
> limit of resource acquirement. In worse case, when new start containers
> consistently fail, YarnResourceManager will goes into an infinite resource
> acquirement process without failing the job. Together with the
> https://issues.apache.org/jira/browse/FLINK-10848, It will quick occupy all
> resources of yarn queue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
