XComp opened a new pull request #18536: URL: https://github.com/apache/flink/pull/18536
Related to [FLIP-194](https://cwiki.apache.org/confluence/display/FLINK/FLIP-194%3A+Introduce+the+JobResultStore) ## What is the purpose of the change We want to introduce a generic interface for cleaning up job-related data in the `Dispatcher`. This affects the `BlobServer`, `HighAvailabilityServices`, `JobManagerMetricsGroup` and `JobGraphStore`. The goal is to move the cleanup logic into `ResourceCleaner` and remove complexity from the `Dispatcher` implementation. ## Brief change log * Introduced interfaces for local and global cleanup * Refactor `BlobServer`, `HighAvailabilityServices`, `JobManagerMetricsGroup` and `JobGraphStore` to implement the new interfaces * Introduce `JobManagerRunnerRegistry` to wrap the HashMap holding the `JobManagerRunner` instances in the `Dispatcher`. This enables us to let `JobManagerRunnerRegistry` implement the new interfaces as well and to have the `JobManagerRunner.close` method being called through the cleanup. * Adds `ResourceCleaner` interface and related helper/test classes * Additionally, some minor hotfixes were added. ## Verifying this change * These changes are mostly covered by existing tests that were just refactored to match the new interfaces * A new unit test `JobManagerRunnerRegistryTest` was added to cover the `JobManagerRunnerRegistry` implementation * We had to adapt the `DispatcherFailoverITCase` for it to succeed in the current state. This test might need a more general refactoring in some follow-up PR. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: yes - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
