Till Rohrmann created FLINK-6804:
------------------------------------
Summary: Inconsistent state migration behaviour between different
state backends
Key: FLINK-6804
URL: https://issues.apache.org/jira/browse/FLINK-6804
Project: Flink
Issue Type: Bug
Components: State Backends, Checkpointing, Type Serialization System
Affects Versions: 1.3.0, 1.4.0
Reporter: Till Rohrmann
Priority: Critical
The {{MemoryStateBackend}}, {{FsStateBackend}} and {{RocksDBStateBackend}} show
a different behaviour when it comes to recovery from old state and state
migration. For example, using the {{MemoryStateBackend}} it is possible to
recover pojos which now have additional fields (at recovery time). The only
caveat is that the recovered {{PojoSerializer}} will silently drop the added
fields when writing the new Pojo. In contrast, the {{RocksDBStateBackend}}
correctly recognizes that a state migration is necessary and thus fails with a
{{StateMigrationException}}. The same applies to the case where Pojo field
types change. The {{MemoryStateBackend}} and the {{FsStateBackend}} accept such
a change as long as the fields still have the same length. The
{{RocksDBStateBackend}} correctly fails with a {{StateMigrationException}}.
I think that all state backends should behave similarly and give the user the
same recovery and state migration guarantees. Otherwise, it could happen that
jobs run with one state backend but not with another (wrt semantic behaviour).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
