Re: [Discuss] State Backend use external HBase storage

Hi Jinkui, Remote state backend is in discussion phase, we send out design for some times. Due to the fact it will be impacted with dynamic scaling and expected non partition state changes, we decided to revisit after dusts settled. Thanks, Chen On Thu, Nov 17, 2016 at 2:23 AM, sjk

Re: [Discuss] State Backend use external HBase storage

Hi, Chen Qin I fount this issue. Does it kicked off? What’s the current progress? https://issues.apache.org/jira/browse/FLINK-4266 > On Nov 16, 2016, at 19:35, Till Rohrmann wrote: > > Hi Jinkui, > > the file system

Re: [Discuss] State Backend use external HBase storage

Hi Jinkui, the file system state backend and the RocksDB state backend can be configured (and usually should be) such that they store their checkpoint data on a reliable storage system such as HDFS. Then you also have the reliability guarantees. Of course, one can start adding more state

[Discuss] State Backend use external HBase storage

Hi, All At present flink have three state backend: memory, file system, rocksdb. MemoryStateBackend will tansform the snapshot to jobManager, 5MB limited default. Even setting it bigger, that not suitable for very big state storage. HDFS can meet the reliability guarantee, but It's slow. File