FloChehab opened a new issue #10541: URL: https://github.com/apache/airflow/issues/10541
**Apache Airflow version**: 1.10.12 rc4 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): v1.16.11-gke.5 **Environment**: - **Cloud provider or hardware configuration**: / - **OS** (e.g. from /etc/os-release): / - **Kernel** (e.g. `uname -a`): / - **Install tools**: / - **Others**: `apache/airflow@sha256:6de1374274f26836c98bbe9f8c065215491f8f5bd48bedc155765dec9b883144` **What happened**: This issue is a followup to discussions on https://github.com/apache/airflow/pull/10230#issuecomment-679274286 . Let's take this dag: ```python from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator from airflow.kubernetes.secret import Secret from airflow.models import DAG from airflow.utils.dates import days_ago default_args = { 'owner': 'Airflow', 'start_date': days_ago(2), 'retries': 3 } with DAG( dag_id='bug_kuberntes_pod_operator', default_args=default_args, schedule_interval=None ) as dag: k = KubernetesPodOperator( namespace='airflow', image="ubuntu:16.04", cmds=["bash", "-cx"], arguments=["sleep 100"], name="airflow-test-pod", task_id="task", get_logs=True, is_delete_operator_pod=True, ) ``` If you: 1. Trigger the dag, 2. Wait for the task to be up and running on kubernetes, 3. Kill everything related to airflow (except the task running on kubernetes), 4. Wait for the task to complete on Kubernetes, 5. Restart airflow. The the task would be marked as `up_for_retry` and would be stuck in this state until another scheduler restart. **What you expected to happen**: The task to be marked as success on the first scheduler restart or not stuck in `up_for_retry` state. **How to reproduce it**: * Use the dag above, * Tested with LocalExecutor and CeleryExecutor (on keda) ; both with helm chart from master. With no major changes except setting the timezone to Europe/Paris. **Anything else we need to know**: * The issue seems to appear every time, * Scheduler logs can be found here: https://github.com/apache/airflow/pull/10230#issuecomment-679304807 & https://github.com/apache/airflow/pull/10230#issuecomment-679314891 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
