[
https://issues.apache.org/jira/browse/FLINK-21688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17311180#comment-17311180
]
Yun Tang edited comment on FLINK-21688 at 3/30/21, 6:40 AM:
------------------------------------------------------------
This is because {{setMaxBackgroundJobs}} is first introduced from RocksDB-5.9 (
[https://github.com/facebook/rocksdb/blob/489d69679a659bef10d58e1e717d628b32ad537c/java/src/main/java/org/rocksdb/DBOptionsInterface.java#L554]
), while Flink adopts RocksDB from 4.x and only having
{{setIncreaseParallelism}} interface. That's why Flink still used old
{{setIncreaseParallelism}} interface at that time. You could also configure
your own {{DBOptions}} to leverage {{setMaxBackgroundJobs}} interface.
BTW, apache JIRA is not somewhere to ask questions, you should better ask
related question at Flink user-mailling list.
was (Author: yunta):
This is because {{setMaxBackgroundJobs}} is first introduced from RocksDB-5.9 (
https://github.com/facebook/rocksdb/blob/489d69679a659bef10d58e1e717d628b32ad537c/java/src/main/java/org/rocksdb/DBOptionsInterface.java#L554
), while Flink adopts RocksDB from 4.x and only having
{{setIncreaseParallelism}} interface. That's why Flink still used old
{{setIncreaseParallelism}} interface. You could also configure your own
{{DBOptions}} to leverage {{setMaxBackgroundJobs}} interface.
BTW, apache JIRA is not somewhere to ask questions, you should better ask
related question at Flink user-mailling list.
> we use setIncreaseParallelism function, can cause slow flush in restore
> -----------------------------------------------------------------------
>
> Key: FLINK-21688
> URL: https://issues.apache.org/jira/browse/FLINK-21688
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Affects Versions: 1.11.3
> Reporter: xiaogang zhou
> Priority: Major
>
> we use setIncreaseParallelism function, can cause slow flush in restore
> rescaling case. As this function limits the HIGH threads to 1.
>
> Why not set the MAX jobs to 40, which will offer more flush thread to enable
> a fast recovery
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
