[
https://issues.apache.org/jira/browse/FLINK-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17028017#comment-17028017
]
Canbin Zheng edited comment on FLINK-15648 at 2/2/20 2:45 AM:
--------------------------------------------------------------
Thanks [~fly_in_gis]. This ticket is about the CPU resource requirement, it
would be good to narrow the scope.
>>> Could you share the key advantages of separate the request and limit
>>> configuration
As you said, one can set the resource limit larger than the request to make the
most of the resources, such as share CPU resources when the machine CPU load is
low. And of course, one can just disable the CPU limit, just like what you do
for the internal use case, so that the Pods could use the CPU resource from the
landing machine as much as possible if they need it. However it’s not always
suitable(also not particularly recommended) to disable the CPU limit in
production, especially for the machine with large specification and has many
Pods running on, because the potential CPU competition brings extra overhead
due to the CPU scheduling mechanism, so in some case, one needs to provide
specific value for the limit configuration option, instead of just disabling
the limit.
As to the limit configuration option, it can satisfy your internal use case
too, the typical usage is as below:
# if not set, fallback to the CPU request configuration option
# if set with a number no more than zero, disable the CPU limit
# set with a number larger than the CPU request configuration option
>>> Since it will make Kubernetes configuration very different from
>>> others(standalone, Yarn, Mesos).
Could you share more details on why it will make K8s configuration very
different from the standalone and Mesos cluster managers?
As to the Yarn cluster manager, it does not support well for the request and
limit CPU settings, so the {{yarn.xxx.vcores}} configuration option is just
good enough.
Lastly, it could be further discussed whether we should deprecate the
{{kubernets.xxx.cpu}} option, but users could be beneficial if we provide a
set entry point for the Pod CPU limit.
was (Author: felixzheng):
Thanks [~fly_in_gis] . This ticket is about the CPU resource requirement, it’s
good to narrow the scope.
>>> Could you share the key advantages of separate the request and limit
>>> configuration
As you said, one can set the resource limit larger than the request to make the
most of the resources, such as share CPU resource when the machine CPU load is
low. And of course, one can just disables the CPU limit, just like what you do
for the internal use case, so that the Pods could use the CPU resource from the
landing machine as much as possible if they need. However it’s not always
suitable(also not particular recommended) to disable the CPU limit in
production, especially for the machine with large specification and has many
Pods running on, because the potential CPU competition brings extra overhead
due to the CPU scheduling mechanism, so in some case one needs to provide
specific value for the limit configuration option, instead of just disabling
the limit.
As to the limit configuration option, it can satisfy your internal use case
also, the typical usage is as below:
# if not set, fallback to the CPU request configuration option
# if set with number no more than zero, disable the CPU limit
# set with number larger than the CPU request configuration option
>>> Since it will make kubernetes configuration very different from
>>> others(standalone, Yarn, mesos).
Could you share more details why it will make K8s configuration very different
from the standalone and mesos cluster managers?
As to the Yarn cluster manager, it does not support well for the request and
limit CPU settings, so the {{yarn.xxx.vcores}} configuration option is just
good enough.
Lastly, it could be further discussed whether we should deprecate the
{{kubernets.xxx.cpu}} configuration option, anyway I propose to provide setting
entrypoint for the CPU limit.
> Support to configure request or limit for CPU requirement
> ---------------------------------------------------------
>
> Key: FLINK-15648
> URL: https://issues.apache.org/jira/browse/FLINK-15648
> Project: Flink
> Issue Type: Sub-task
> Components: Deployment / Kubernetes
> Reporter: Canbin Zheng
> Priority: Major
>
> The current branch use kubernetes.xx.cpu to configure request and limit
> resource requirement for a Container, it's an important improvement to
> separate these two configurations, we can use kubernetes.xx.request.cpu and
> kubernetes.xx.limit.cpu to specify request and limit resource
> requirements.{color:#6a8759}
> {color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
