[
https://issues.apache.org/jira/browse/FLINK-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087666#comment-17087666
]
Xintong Song commented on FLINK-15648:
--------------------------------------
[~felixzheng], sorry to jump into the discussion late.
I agree with you that it is a good thing to leverage the Kubernetes feature of
container request/limit cpu. However, I'm not entirely sure about the proposed
approach that introduces configuration options for the absolute cpu limit for
JM/TM containers.
It might be ok with the current Flink resource management, where we have only
two kinds of containers. But it may not work well with FLINK-14106, where we no
longer assumes that all the TMs have the same amount of resources. With the
efforts towards fine grained resource management, we might eventually have
SlotManager plugin that dynamically decides TM resources based on the workload.
In such cases, we may not have a proper `kubernetes.taskmanager.limit.cpu`
value that works well with all TMs with different resources, and the
SlotManager plugin should not be aware of the specific underlying resource
manager (K8s, Yarn or Mesos) and decide the Kubernetes specific 'limit.cpu'.
Alternatively, we might have a configuration option
'kubernetes.container.cpu-limit-ratio' or so, that calculate the 'limit.cpu'
from the 'request.cpu'. E.g., if the requested cpu is 2.0 and the ratio is 1.5,
then the limit would be 3.0. I think this gives you practically the same
controllability as the proposed approach, while naturally works with dynamic TM
resources.
WDYT?
> Support to configure limit for CPU requirement
> ----------------------------------------------
>
> Key: FLINK-15648
> URL: https://issues.apache.org/jira/browse/FLINK-15648
> Project: Flink
> Issue Type: Sub-task
> Components: Deployment / Kubernetes
> Reporter: Canbin Zheng
> Assignee: Canbin Zheng
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The current branch use kubernetes.xx.cpu to configure request and limit
> resource requirement for a Container, it's an important improvement to
> separate these two configurations, we can use kubernetes.xx.request.cpu and
> kubernetes.xx.limit.cpu to specify request and limit resource
> requirements.{color:#6a8759}
> {color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
