[
https://issues.apache.org/jira/browse/FLINK-35501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Khachatryan closed FLINK-35501.
-------------------------------------
Release Note: The semantics of
state.backend.rocksdb.checkpoint.transfer.thread.num changed slightly: If
negative, the common (TM) IO thread pool is used (see cluster.io-pool.size) for
up/downloading RocksDB files
Resolution: Fixed
> Use common thread pools when transferring RocksDB state files
> -------------------------------------------------------------
>
> Key: FLINK-35501
> URL: https://issues.apache.org/jira/browse/FLINK-35501
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Affects Versions: 1.20.0
> Reporter: Roman Khachatryan
> Assignee: Roman Khachatryan
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
>
> Currently, each RocksDB state backend creates an executor backed by a thread
> pool.
> This makes it difficult to control the total number of threads per TM because
> it might have at least one task per slot and theoretically, many state
> backends per task (because of chaining).
> Additionally, using a common thread pool allows to indirectly control the
> load on the underlying DFS (e.g. the total number of requests to S3 from a
> TM).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
