Myasuka commented on a change in pull request #11491:
[FLINK-16513][checkpointing] Unaligned checkpoints: checkpoint metadata
URL: https://github.com/apache/flink/pull/11491#discussion_r399968786
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/metadata/MetadataV2V3SerializerBase.java
##########
@@ -212,46 +218,66 @@ protected MasterState
deserializeMasterState(DataInputStream dis) throws IOExcep
//
------------------------------------------------------------------------
protected void serializeSubtaskState(OperatorSubtaskState subtaskState,
DataOutputStream dos) throws IOException {
- final OperatorStateHandle managedOperatorState =
extractSingleton(subtaskState.getManagedOperatorState());
- if (managedOperatorState != null) {
+ serializeSingleton(subtaskState.getManagedOperatorState(), dos,
MetadataV2V3SerializerBase::serializeOperatorStateHandle);
+ serializeSingleton(subtaskState.getRawOperatorState(), dos,
MetadataV2V3SerializerBase::serializeOperatorStateHandle);
+ serializeKeyedStateCol(subtaskState.getManagedKeyedState(),
dos);
+ serializeKeyedStateCol(subtaskState.getRawKeyedState(), dos);
+ serializeCollection(subtaskState.getInputChannelState(), dos,
this::serializeInputChannelStateHandle);
+ serializeCollection(subtaskState.getResultSubpartitionState(),
dos, this::serializeResultSubpartitionStateHandle);
Review comment:
I noticed something weird here. Although we would not use
`MetadataV2Serializer` to serialize checkpoint after this version, however, you
actually changed the serialized format of `MetadataV2Serializer`, it would
serialize two `0` into the stream here. On the other hand,
`MetadataV2Serializer#deserializeSubtaskState` would ignore the written part
but just return a `StateObjectCollection.empty()`.
In other words, this PR actually changed the logic of serialize and
deserialize of `MetadataV2Serializer`. Although we might not meet this error in
real environment as we would always use `MetadataV3Serializer` to serialize.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
