GitHub user potiuk added a comment to the discussion: Should running KubernetesPodOperator pods terminate on Airflow scheduler restart?
> If it is expected, is it possible to prevent KPO pods in the running state > from termination on scheduler restart (without hacks like custom on_kill() > implementation)? Yes. It's expected and it's consequence of using Local Executor. If you want the Pods to not be terminated you need to use Celery Executor or KubernetesExecutor to run KPOs.The reasons why it is like that is that executors are running the tasks even if KPO is used. LocalExecutor is - by definition and architectur e- restarted when you restart Scheduler and for Local Executor the tasks are "subprocesses" of the executore. In case of Celery and Kubernetes executor, tasks are running "somewwhere else" - either as separate worker subprocesses or separate k8S pods, so they "survive" scheduler restarts. So really the docs are right - it's not the operator used that determines survivability in this case - it's the executor. GitHub link: https://github.com/apache/airflow/discussions/61734#discussioncomment-15759792 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
