[ https://issues.apache.org/jira/browse/FLINK-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
LiuBiao closed FLINK-4164. -------------------------- Resolution: Won't Fix As the state backend has been reconstructed, this issue should be closed. > Use only one rocksdb column family to store all states in an operator > --------------------------------------------------------------------- > > Key: FLINK-4164 > URL: https://issues.apache.org/jira/browse/FLINK-4164 > Project: Flink > Issue Type: Improvement > Components: State Backends, Checkpointing > Reporter: LiuBiao > Priority: Minor > > I see now in master branch, there is only one rocksdb instance in an > operator, each state will be assigned into separated column families. It's a > good improvement, but I think there are still some problems. Column families > in rocksdb do not share memtable and sst files(correct me if I am wrong, > there is not much docs about column families in rocksdb). If user have > thousands of states, the overhead will be costly. Memory may will be > exhausted, also there will be too much small sst files. > If we use prefix in key instead of column family, the memory that rocksdb > costed will be under control. The shortcoming is that we can not optimize > states separately, and performance may be a little worse than before. But I > think it is worth to do, it is meaning for some cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)