[
https://issues.apache.org/jira/browse/FLINK-37832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17953551#comment-17953551
]
Zhenqiu Huang commented on FLINK-37832:
---------------------------------------
The resource-requirements rest api is used in autoscaler of Flink operator to
scale up and down Flink jobs. After applying the autoscaling to Flink streaming
workload, we find that scale up parallelism will help to quickly reduce the
kafka lag, but there is a side effect of creating smaller files. The best the
solution for ingestion use cases is that we can adjust the parallelism together
with flink checkpoint interval. With these two parameter adjustment, we can
control the files size with smart autoscaler algorithm.
> Allow update checkpoint interval in update resource-requirements api
> --------------------------------------------------------------------
>
> Key: FLINK-37832
> URL: https://issues.apache.org/jira/browse/FLINK-37832
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 2.1.0
> Reporter: Zhenqiu Huang
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
