igalshilman commented on a change in pull request #7580: [FLINK-11436] Fix Java
deserialization failure of the AvroSerializer
URL: https://github.com/apache/flink/pull/7580#discussion_r251403045
##########
File path:
flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/typeutils/AvroSerializer.java
##########
@@ -381,4 +383,76 @@ public AvroSchemaSerializerConfigSnapshot() {
}
}
+
+ // -------- backwards compatibility with 1.6 -----------
+
+ private void readObject(ObjectInputStream in) throws IOException,
ClassNotFoundException {
+ /*
+ During the release of Flink 1.7, the value of serialVersionUID
was uptick to 2L (was 1L before)
+ And although the AvroSerializer (along with it's snapshot
class) were migrated to the new serialization
+ abstraction (hence free from Java serialization), there were
composite serializers that were not migrated
+ and were serialized with Java serialization. In case that one
of the nested serializers were Avro we would
+ bump into deserialization exception due to a wrong
serialVersionUID. Unfortunately it is not possible to revert
+ the serialVersionUID back to 1L, because users might have
snapshots with 2L present already.
+ To overcome this we first need to make sure that the
AvroSerializer is being Java deserialized with
+ FailureTolerantObjectInputStream, and then we determine the
serialized layout by looking at the fields.
+
+ From:
https://docs.oracle.com/javase/8/docs/platform/serialization/spec/class.html#a5421
+
-------------------------------------------------------------------------------------------------------------
+ The descriptors for primitive typed fields are written first
+ sorted by field name followed by descriptors for the object
typed fields sorted by field name.
+ The names are sorted using String.compareTo.
+
-------------------------------------------------------------------------------------------------------------
+
+ pre 1.7 field order: [schemaString, type]
Review comment:
Good point! apparently Flink 1.5 had a different layout, ae70a99c94 adds
support to that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
