The GitHub Actions job "Tests (AMD)" on airflow.git/k8s-executor-lazy-manager has failed. Run started by GitHub user kaxil (triggered by kaxil).
Head commit for run: a3ebff1a514788d8227a220773b17f6341346465 / Kaxil Naik <[email protected]> Fix KubernetesExecutor leaking a Manager process when reading running task logs The API server constructs a KubernetesExecutor solely to call get_task_log() for RUNNING tasks (via FileTaskHandler -> ExecutorLoader.get_default_executor) and never starts or ends it. KubernetesExecutor.__init__ eagerly created a multiprocessing.Manager(), which forks a serve_forever child process. Because that instance is cached per process and never shut down, the Manager child is orphaned -- one leaked process (~350-400 MB resident) per API-server worker, growing with worker recycling and pushing the API server toward OOM. get_task_log() only needs the kube client and pod namespace; it never touches the task/result queues. Create the Manager and its queues lazily in start() (the scheduling loop is their only consumer), mirroring how LocalExecutor already defers process/queue creation. end() now no-ops when the executor was never started. Constructing the executor for log reading no longer spawns a Manager. Report URL: https://github.com/apache/airflow/actions/runs/27923953375 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
