[
https://issues.apache.org/jira/browse/FLINK-11869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-11869:
-----------------------------------
Labels: pull-request-available (was: )
> [checkpoint] Make buffer size in checkpoint stream factory configurable
> -----------------------------------------------------------------------
>
> Key: FLINK-11869
> URL: https://issues.apache.org/jira/browse/FLINK-11869
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Checkpointing
> Reporter: Yun Tang
> Assignee: Yun Tang
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.9.0
>
>
> Currently, the default buffer size for {{FsCheckpointStateOutputStream}} is
> only 4KB. This would case a lot of IOPS if stream is large. Unfortunately,
> when user want to checkpoint on a totally disaggregated file system which has
> no data node manager running in local machine, they might have a IOPS limit
> or cannot serve too many IOPS at a time. This would cause the checkpoint
> duration really large and might expire often.
> If we want to increase this buffer size, we have to increase the
> {{fileStateThreshold}} to indirectly increase the buffer size. However, as we
> all know, too many not-so-small {{ByteStreamStateHandle}} returned to
> checkpoint coordinator would easily cause job manager OOM and checkpoint meta
> file large.
> We should also make the buffer size configurable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
