[
https://issues.apache.org/jira/browse/FLINK-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhijiang updated FLINK-5138:
----------------------------
Description:
Users may specify the state size in *setResources* API, then the state backend
would roughly estimate the different kinds of memory usages based on the state
size. This part of estimate memory would be aggregated with other memories in
*ResourceSpec*.
There are two advantages to do this:
- For *RocksDB* backend, the proper memory setting for *RocksDB* may get
better performance in read and write operation.
- This estimate memory will be considered together when requesting resource
for container, so the total memory usage will not exceed the container limit.
was:
Users may specify the state size in *setResource* API, then the different
*StateBackend* implementation should roughly estimate the different kinds of
memory usages based on the state size. This part of estimate memory will be
aggregated with other memories in *ResourceSpec*. There are two advantages to
do this:
- For RocksDB backend, the proper memory setting for RocksDB can get better
performance in read and write.
- This estimate memory will be considered when requesting resource for
container, so the total memory usage will not exceed the container limit.
> Add the interface in state backend to estimate memory by hint of state size
> in ResourceSpec
> -------------------------------------------------------------------------------------------
>
> Key: FLINK-5138
> URL: https://issues.apache.org/jira/browse/FLINK-5138
> Project: Flink
> Issue Type: Sub-task
> Components: State Backends, Checkpointing
> Reporter: zhijiang
> Assignee: zhijiang
>
> Users may specify the state size in *setResources* API, then the state
> backend would roughly estimate the different kinds of memory usages based on
> the state size. This part of estimate memory would be aggregated with other
> memories in *ResourceSpec*.
> There are two advantages to do this:
> - For *RocksDB* backend, the proper memory setting for *RocksDB* may get
> better performance in read and write operation.
> - This estimate memory will be considered together when requesting resource
> for container, so the total memory usage will not exceed the container limit.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
