[
https://issues.apache.org/jira/browse/FLINK-8716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370052#comment-16370052
]
Arvid Heise commented on FLINK-8716:
------------------------------------
In the email thread, [~till.rohrmann] also pointed out that
cancel-with-snapshot always produces full checkpoints. So within a snapshot
only one serializer version is used.
The question should be: what happens when we do an incremental checkpoint
afterwards. Then we probably need two different serializer configurations for
the same descriptor. That sounds doable (use descriptor name/checkpoint name as
the key of the meta information instead of just the descriptor name) but may
require larger refactoring.
> AvroSerializer does not use schema of snapshot
> ----------------------------------------------
>
> Key: FLINK-8716
> URL: https://issues.apache.org/jira/browse/FLINK-8716
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.4.0
> Reporter: Arvid Heise
> Priority: Major
>
> The new AvroSerializer stores the schema in the snapshot and uses it to
> validate compability.
> However, it does not use the schema of the snapshot while reading the data.
> This version will fail for any change of the data layout (so it supports more
> or less only renaming currently).
>
> [https://github.com/apache/flink/blob/f3a2197a23524048200ae2b4712d6ed833208124/flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/typeutils/AvroSerializer.java#L265]
> needs to use the schema from
>
> [https://github.com/apache/flink/blob/f3a2197a23524048200ae2b4712d6ed833208124/flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/typeutils/AvroSerializer.java#L188]
> as the first parameter. Accordingly, a readSchema field need to be set
> in #ensureCompatibility and relayed in #duplicate. Note that the readSchema
> is passed as the write schema parameter to the DatumReader, as it was the
> schema that was used to write the data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
