XComp opened a new pull request, #23773: URL: https://github.com/apache/flink/pull/23773
## What is the purpose of the change This is a follow-up of [PR #23415](https://github.com/apache/flink/pull/23415) which tries to come up with a more general solution for working around the thread leakage on the ZooKeeper server side. The motivation was that the approach which was introduced in #23415 changed some visibility constraints that are not really desired (making the `ZooKeeperLeaderRetrievalDriver` aware of the `ResourceManager`). This change is meant as a draft and should rather be used as a base for a discussion whether FLINK-33053 is actually an issue that should be addressed in Flink itself. As far as I understand, the thread leakage is only happening in test code when the ZooKeeper test server implementation is set up. ## Brief change log * Introduced a reference counter in the `ZooKeeperLeaderRetrievalFactory` that counts the number of instances that are created by this factory instance. This reference counter is passed into all driver instances and is used to decide whether the `removeAll` call should be performed during `close()`. ## Verifying this change I tried to come up with a test to verify the behavior. But it's hard to test because the thread leakage is actually happening in the ZooKeeper server itself. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: yes - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
