[
https://issues.apache.org/jira/browse/FLINK-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007367#comment-17007367
]
Yun Tang commented on FLINK-15368:
----------------------------------
[~liyu] thanks for you summary, I agreed that we should add a buffer among the
caculated off-heap memory or user configured fixed memory
{{state.backend.rocksdb.memory.fixed-per-slot}} and allocate the cache with
memory of 3/(3 + write_ratio) * expected_memory. If memory used by pinned block
cache or iterator caused the OOM, user could adjust
{{taskmanager.memory.task.off-heap.size}} to add more buffer there.
Last but not least, if so, we should add more description within our doc PR
[https://github.com/apache/flink/pull/10498]
I'll also implement local tests with above strategy to crate the block cache to
see anything unexpected happened.
> Add end-to-end test for controlling RocksDB memory usage
> --------------------------------------------------------
>
> Key: FLINK-15368
> URL: https://issues.apache.org/jira/browse/FLINK-15368
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / State Backends
> Affects Versions: 1.10.0
> Reporter: Yu Li
> Assignee: Yun Tang
> Priority: Critical
> Fix For: 1.10.0
>
>
> We need to add an end-to-end test to make sure the RocksDB memory usage
> control works well, especially under the slot sharing case.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
