Dennis-Mircea Ciupitu created FLINK-41031:
---------------------------------------------
Summary: Close operator-owned executors and metric reporters on
operator shutdown
Key: FLINK-41031
URL: https://issues.apache.org/jira/browse/FLINK-41031
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Dennis-Mircea Ciupitu
h1. Problem
The operator only stops two components on shutdown: the JOSDK {{Operator}} (via
{{installShutdownHook}}) and {{OperatorHealthService}} (via its own shutdown
hook, registered only in {{run()}}). {{FlinkOperator.stop()}} calls just
{{operator.stop()}} and is only used for tests, not as a shutdown hook for the
operator itself.
Everything else the operator creates is never closed:
* {{FlinkConfigManager}}: a non-daemon scheduler for config cache cleanup and
the dynamic config watcher
* {{CanaryResourceManager}}: a non-daemon scheduled pool per controller
* {{FlinkResourceContextFactory}}: the {{Flink-RestClusterClient-IO}} pool
* The operator {{MetricRegistry}} and its reporters
h1. Impact
* On pod termination, the config watcher and canary checks keep firing while
JOSDK drains in-flight reconciliations. In-flight REST calls on the IO pool are
cut off instead of drained. Metric reporters are never closed, so push-based
reporters can lose their last flush.
* {{FlinkOperator.stop()}} leaves non-daemon threads running, so an embedded or
test operator does not release its resources. In tests, every
{{FlinkConfigManager}} leaks a scheduler thread that keeps the manager and its
config cache reachable for the rest of the surefire fork.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
