Github user tzulitai commented on the issue:
https://github.com/apache/flink/pull/3031
A re-clarification about backwards compatibility for state type change:
Currently, one limitation for compatible applications across savepoint
restore is that you can't change the type of state otherwise state restore will
fail, therefore not compatible. The only work around, is to have another field
as the new state with the new type, and somehow try to "encode" / "decode" the
watermark state into / from the original `Tuple2<KafkaTopicPartition, Long>`. I
don't think this is easily possible ...
At the same time, there was recent discussion about letting the window
operators also checkpoint watermarks. So perhaps we might not need to let the
Kafka sources checkpoint watermarks in the end, if the window operators already
take care of restoring the previous event time.
What I'm curious about right now is whether or not in the future,
redistributions of Kafka partition states across source subtasks will work well
with the checkpointed watermarks in the downstream window operators. I don't
think there should be a problem.
@aljoscha can you perhaps help clarify this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
