Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/4301
Yes, I don't think we can get around this when restoring from "old" state.
I also have another suspicion: I don't think that
`KafkaConsumerTestBase.runMultipleSourcesOnePartitionExactlyOnceTest()`
accurately catches some cases and I think there is a problem that we cannot
accurately detect whether we are restoring or whether we are opening from
scratch. Consider this case: 5 partitions, 5 parallel source instances. Now we
rescale to 10 parallel source instances. Some sources don't get state, so they
think that we are starting from scratch and they will run partition discovery.
Doesn't this mean that they could possibly read from a topic where already
another source is reading from, because it got the state for that? (Not this
doesn't occur on master because all sources get all state.)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
