[
https://issues.apache.org/jira/browse/FLINK-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LiuBiao closed FLINK-4164.
--------------------------
Resolution: Won't Fix
As the state backend has been reconstructed, this issue should be closed.
> Use only one rocksdb column family to store all states in an operator
> ---------------------------------------------------------------------
>
> Key: FLINK-4164
> URL: https://issues.apache.org/jira/browse/FLINK-4164
> Project: Flink
> Issue Type: Improvement
> Components: State Backends, Checkpointing
> Reporter: LiuBiao
> Priority: Minor
>
> I see now in master branch, there is only one rocksdb instance in an
> operator, each state will be assigned into separated column families. It's a
> good improvement, but I think there are still some problems. Column families
> in rocksdb do not share memtable and sst files(correct me if I am wrong,
> there is not much docs about column families in rocksdb). If user have
> thousands of states, the overhead will be costly. Memory may will be
> exhausted, also there will be too much small sst files.
> If we use prefix in key instead of column family, the memory that rocksdb
> costed will be under control. The shortcoming is that we can not optimize
> states separately, and performance may be a little worse than before. But I
> think it is worth to do, it is meaning for some cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
