Re: Migrating from one state backend to another

[Josh]

Hi Aljoscha,
Thanks, that makes sense. I will start using RocksDB right away then.

Josh

On Wed, Jun 15, 2016 at 1:01 PM, Aljoscha Krettek 
wrote:

> Hi,
> right now migrating from one state backend to another is not possible. I
> have it in the back of my head, however, that we should introduce a common
> serialized representation of state to make this possible in the future.
> (Both for checkpoints and savepoints, which use the same mechanism
> underneath.)
>
> Cheers,
> Aljoscha
>
> On Wed, 15 Jun 2016 at 00:04 Josh  wrote:
>
>> I'm trying to decide whether to set-up RocksDB now or later.
>>
>> The state for my Flink jobs right now will be able to fit into memory, so
>> I can use the filesystem state backend. In a few months time it is likely
>> the state will no longer fit into memory, so I will want to use the RocksDB
>> backend.
>>
>> I was just wondering if it's possible/easy to use savepoints to migrate
>> existing state from the filesystem backend to the RocksDB backend? As I
>> would not want to lose any job state when switching to RocksDB. If there's
>> a way to do it then I can worry about RocksDB later.
>>
>> Thanks!
>> Josh
>>
>

Re: Migrating from one state backend to another

[Aljoscha Krettek]

Hi,
right now migrating from one state backend to another is not possible. I
have it in the back of my head, however, that we should introduce a common
serialized representation of state to make this possible in the future.
(Both for checkpoints and savepoints, which use the same mechanism
underneath.)

Cheers,
Aljoscha

On Wed, 15 Jun 2016 at 00:04 Josh  wrote:

> I'm trying to decide whether to set-up RocksDB now or later.
>
> The state for my Flink jobs right now will be able to fit into memory, so
> I can use the filesystem state backend. In a few months time it is likely
> the state will no longer fit into memory, so I will want to use the RocksDB
> backend.
>
> I was just wondering if it's possible/easy to use savepoints to migrate
> existing state from the filesystem backend to the RocksDB backend? As I
> would not want to lose any job state when switching to RocksDB. If there's
> a way to do it then I can worry about RocksDB later.
>
> Thanks!
> Josh
>