Cassandra backend would be interesting especially if flink could benefit from cassandra data locality. Cassandra/spark integration is using this for information to schedule spark tasks.
On 9 June 2016 at 19:55, Nick Dimiduk <ndimi...@gmail.com> wrote: > You might also consider support for a Bigtable > backend: HBase/Accumulo/Cassandra. The data model should be similar > (identical?) to RocksDB and you get HA, recoverability, and support for > really large state "for free". > > On Thursday, June 9, 2016, Chen Qin <qinnc...@gmail.com> wrote: > > > Hi there, > > > > What is progress on incremental checkpointing? Does flink dev has plan to > > work on this or JIRA to track this? super interested to know. > > > > I also research and consider use rocksdbstatebackend without running HDFS > > cluster nor talk to S3. Some primitive idea is to use ZK to store / > notify > > state propagation progress and propagate via implement chain replication > on > > top of YARN provisioned storage node. > > > > Thanks, > > Chen > > >
