[
https://issues.apache.org/jira/browse/FLINK-15692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022829#comment-17022829
]
Yun Tang commented on FLINK-15692:
----------------------------------
I believe the performance regression is because micro performance benchmark
does not give explicit managed memory configuration. Since micro benchmark use
mini cluster to launch jobs, and the managed memory for non-explicit
configuration would only be 16MB (refer to [code
implementation|https://github.com/apache/flink/blob/a6c7e6adb1c69b5f226bba49687069975ba231ee/flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniClusterConfiguration.java#L110]).
Since RocksDB has the default basic arena block size as 8MB, 16MB would be
really not enough for write buffer. I suggest to set the managed memory as
322122552 bytes, which is the default managed memory of 1GB TM with 1 slot.
> Enable limiting RocksDB memory consumption by default
> -----------------------------------------------------
>
> Key: FLINK-15692
> URL: https://issues.apache.org/jira/browse/FLINK-15692
> Project: Flink
> Issue Type: Task
> Components: Runtime / State Backends
> Affects Versions: 1.10.0
> Reporter: Till Rohrmann
> Assignee: Yun Tang
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Attachments: RocksStateBackendBenchmark_result_after_this_change.png
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Enable limiting RocksDB memory consumption by default. This means to change
> the default value of {{state.backend.rocksdb.memory.managed}} to {{true}}.
> This will change Flink's default behaviour for many users but I believe the
> benefits of a much more predictable memory behaviour of Flink will outweigh
> the downside of having fewer memory available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
