Re: Migrating from one state backend to another

2016-06-15 Thread Josh
Hi Aljoscha, Thanks, that makes sense. I will start using RocksDB right away then. Josh On Wed, Jun 15, 2016 at 1:01 PM, Aljoscha Krettek <aljos...@apache.org> wrote: > Hi, > right now migrating from one state backend to another is not possible. I > have it in the back of m

Re: Migrating from one state backend to another

2016-06-15 Thread Aljoscha Krettek
Hi, right now migrating from one state backend to another is not possible. I have it in the back of my head, however, that we should introduce a common serialized representation of state to make this possible in the future. (Both for checkpoints and savepoints, which use the same mechanism

Migrating from one state backend to another

2016-06-14 Thread Josh
I'm trying to decide whether to set-up RocksDB now or later. The state for my Flink jobs right now will be able to fit into memory, so I can use the filesystem state backend. In a few months time it is likely the state will no longer fit into memory, so I will want to use the RocksDB backend. I