[
https://issues.apache.org/jira/browse/FLINK-36022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17872234#comment-17872234
]
Rui Fan commented on FLINK-36022:
---------------------------------
Thanks [~heigebupahei] for reporting this issue and for the ping.
{quote}We have enabled AUTOSCALER in some scenarios, but we have not enabled
job.autoscaler.scaling.enabled because we only want to use AUTOSCALER to obtain
resource recommendations for tasks
{quote}
When we started experimenting with the autoscaler, we also disable the
job.autoscaler.scaling.enabled to observe the recommended result.
In addition to the configuration options you listed, I also encountered some
other problems observing recommended parallelism from event handler. (Our
internal autoscaler made some patches for it.)
After enabling the scaling, I found the autoscaler with scaling is more
powerful, and it has a lot of differences with disabling the scaling. It seems
our internal patches are not useful.
In the long term, the recommended parallelism of event handler is only a side
feature(assisted manual check), autoscaler hopes these jobs enable scaling. I'm
not sure is it needed to made more efforts for the case when scaling is disable.
> When scaling.enabled =false, adjust the values of some parameters to
> provide bette recommendation values.
> -------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-36022
> URL: https://issues.apache.org/jira/browse/FLINK-36022
> Project: Flink
> Issue Type: Improvement
> Components: Autoscaler
> Reporter: yuanfenghu
> Priority: Major
>
> h1. Background
> We have enabled AUTOSCALER in some scenarios, but we have not enabled
> job.autoscaler.scaling.enabled because we only want to use AUTOSCALER to
> obtain resource recommendations for tasks, but some parameters can cause
> these recommendations to be inaccurate.
> example ：
> * job.autoscaler.scale-down.max-factor
> If set to 0.5, it means that the vertex can be reduced to up to 50% of the
> original value during scaling. If we do not turn on the
> job.autoscaler.scaling.enabled parameter, then the recommended value here
> will only be 100 for a vertex with 200 parallelism. But in fact, this may
> only require 50 or even lower resources during low periods.
> * job.autoscaler.restart.time
> This parameter will cause the restart event to be used to calculate the
> resources required to chase data during expansion, resulting in the
> recommended resources being too large. However, if
> job.autoscaler.scaling.enabled =false, the restart time will be 0
>
> h1. Solution:
>
> When job.autoscaler.scaling.enabled = false, actively modify the above
> parameters job.autoscaler.scale-down.max-factor=1,
> job.autoscaler.restart.time=0
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
