wyndhblb commented on issue #6602: [AIRFLOW-6012] - performance improvement to cleanup URL: https://github.com/apache/airflow/pull/6602#issuecomment-557585997 Yes that is correct, and is the current behavior in the "main release" (1.10.6). should https://github.com/apache/airflow/blob/master/airflow/executors/kubernetes_executor.py#L781 fail to find the pod/tasks in it's queue, the KeyError will trap it, and simply move on. We have experienced ~20min speed up in general scheduling with this small change, having hundreds of tasks that complete every 10-15min makes that DB loop prohibitively expensive for every done task.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
