tzulitai commented on a change in pull request #7020: [FLINK-10774] [Kafka]
connection leak when partition discovery is disabled an…
URL: https://github.com/apache/flink/pull/7020#discussion_r246082400
##########
File path:
flink-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumerBase.java
##########
@@ -732,9 +745,7 @@ public void run() {
throw new RuntimeException(discoveryLoopError);
}
} else {
- // won't be using the discoverer
- partitionDiscoverer.close();
-
+ // partitionDiscoverer is already closed in open method
Review comment:
Note that it isn't safe to directly close the partition discoverer if it is
being used by the discovery thread, as it isn't thread-safe (specifically, the
Kafka client isn't thread safe). Would need to consider that when cleaning up
the partition discoverer in the finally block of `kafkaFetcher.runFetchLoop()`.
Please see the comments in the `FlinkKafkaConsumerBase.cancel()` method for
an example.
If the partition discoverer is concurrently being used by the discovery
thread, it must be invoked `wakeup()` instead.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
