tzulitai commented on a change in pull request #7726: [FLINK-10342] Filter
restored partitions with discovered partitions b…
URL: https://github.com/apache/flink/pull/7726#discussion_r260100624
##########
File path:
flink-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumerBase.java
##########
@@ -494,6 +514,12 @@ public void open(Configuration configuration) throws
Exception {
}
}
+ if (filterRestoredPartitionsWithDiscovered) {
+
subscribedPartitionsToStartOffsets.entrySet().removeIf(
+ entry ->
(!allPartitions.contains(entry.getKey()))
Review comment:
Yes, if we use the `topicsDescriptor`, it'll likely require a bit more code.
Perhaps we can add a method:
`boolean KafkaTopicsDescriptor#isMatchingTopic(String)`, which essentially
just does:
```
if (isFixedTopics()) {
return fixedTopics.contain(topic)
} else {
return topicPattern.matcher(topic).matches()
}
```
Can maybe even use this new method to simplify `AbstractPartitionDiscoverer`
a bit.
Either way, this would be a preliminary code refactoring. For this kind of
changes, I suggest having a separate commit in the PR before the actual change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
