[
https://issues.apache.org/jira/browse/FLINK-5397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaogang Shi updated FLINK-5397:
--------------------------------
Description:
To restore from the savepoints in previous versions, Flink now keeps all
classes whose serialization is changed and put them in a separated package
("migration").
When deserializing the old savepoints, flink will look up correct descriptors
({{ObjectStreamClass}}) for these classes, without using those ones written in
serialized data. The implementation however is problematic when there exist
missing field descriptors in the serialized data.
When serializing an object, Java will only write the descriptors of those
non-null fields. But when we look up class descriptors with given classes, all
fields will be put into the descriptors. As a result, we will deserialize the
savepoints with incorrect descriptors, leading to serialization exceptions.
A simple resolution is to update the name of read descriptors using
Reflections, without using different descriptors.
was:
To restore from the savepoints in previous versions, Flink now keeps all
classes whose serialization is changed and put them in a separated package
("migration").
When deserializing the old savepoints, flink will look up correct descriptors
({{ObjectStreamClass}}) for these classes, without using those ones written in
serialized data. The implementation however is problematic when there exist
missing field descriptors in the serialized data.
When deserializing an object, Java will only write the descriptors of those
non-null fields. But when we look up class descriptors with given classes, all
fields will be put into the descriptors. As a result, we will deserialize the
savepoints with incorrect descriptors, leading to serialization exceptions.
A simple resolution is to update the name of read descriptors using
Reflections, without using a different descriptors.
> Fail to deserialize savepoints in v1.1 when there exist missing fields in
> class serialization descriptors
> ---------------------------------------------------------------------------------------------------------
>
> Key: FLINK-5397
> URL: https://issues.apache.org/jira/browse/FLINK-5397
> Project: Flink
> Issue Type: Bug
> Components: State Backends, Checkpointing
> Reporter: Xiaogang Shi
>
> To restore from the savepoints in previous versions, Flink now keeps all
> classes whose serialization is changed and put them in a separated package
> ("migration").
> When deserializing the old savepoints, flink will look up correct descriptors
> ({{ObjectStreamClass}}) for these classes, without using those ones written
> in serialized data. The implementation however is problematic when there
> exist missing field descriptors in the serialized data.
> When serializing an object, Java will only write the descriptors of those
> non-null fields. But when we look up class descriptors with given classes,
> all fields will be put into the descriptors. As a result, we will deserialize
> the savepoints with incorrect descriptors, leading to serialization
> exceptions.
> A simple resolution is to update the name of read descriptors using
> Reflections, without using different descriptors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
