Tzu-Li (Gordon) Tai created FLINK-6190:
------------------------------------------
Summary: Write "Serializer Configurations" metainfo along with
state
Key: FLINK-6190
URL: https://issues.apache.org/jira/browse/FLINK-6190
Project: Flink
Issue Type: Sub-task
Components: State Backends, Checkpointing, Type Serialization System
Reporter: Tzu-Li (Gordon) Tai
Assignee: Tzu-Li (Gordon) Tai
In order for serializers to be able to be reconfigured on restore, we need
knowledge of the previous serializer configurations, e.g. what types were
registered, with which specific / default serializers, and especially for Kryo,
the order they were registered.
For this, we will need serializer configuration metainfo to be self-contained
within the written state.
For the implementation, we propose the following:
- Have a new separate `SerializersConfig` class that is extracted from
`ExecutionConfig`. This new class should contain only the serializer-related
configurations (e.g., `registeredKryoTypes`, `registeredPojoTypes`, etc.).
- Introduce `SerializersConfigSerializationProxy`, which is in charge of
serializing the current `SerializersConfig` when writing state to streams. This
proxy defines the the serialized format of serializer configurations, therefore
should be a `VersionedIOReadableWritable` as we may change the format /
information written in the future.
- Add `SerializersConfigSerializationProxy` into state backends serialization
proxies (e.g. `KeyedBackendSerializationProxy`) so that the serializer
configuration is written into state. Need to additionally make sure backwards
compatibility of previous-version backend serialization proxies.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
