MatrixManAtYrService opened a new issue #17901: URL: https://github.com/apache/airflow/issues/17901
### Apache Airflow version 2.1.3 (latest released) ### Operating System 20.04.1-Ubuntu ### Versions of Apache Airflow Providers Providers info apache-airflow-providers-amazon | 2.1.0 apache-airflow-providers-celery | 2.0.0 apache-airflow-providers-cncf-kubernetes | 2.0.2 apache-airflow-providers-docker | 2.1.0 apache-airflow-providers-elasticsearch | 2.0.2 apache-airflow-providers-ftp | 2.0.0 apache-airflow-providers-google | 5.0.0 apache-airflow-providers-grpc | 2.0.0 apache-airflow-providers-hashicorp | 2.0.0 apache-airflow-providers-http | 2.0.0 apache-airflow-providers-imap | 2.0.0 apache-airflow-providers-microsoft-azure | 3.1.0 apache-airflow-providers-mysql | 2.1.0 apache-airflow-providers-postgres | 2.0.0 apache-airflow-providers-redis | 2.0.0 apache-airflow-providers-sendgrid | 2.0.0 apache-airflow-providers-sftp | 2.1.0 apache-airflow-providers-slack | 4.0.0 apache-airflow-providers-sqlite | 2.0.0 apache-airflow-providers-ssh | 2.1.0 ### Deployment Official Apache Airflow Helm Chart ### Deployment details Dockerfile: ``` FROM apache/airflow:2.1.3 USER root RUN apt update && apt install -y curl # kubectl RUN curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" RUN install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl # kubeconfig RUN mkdir -p ~/.kube COPY ./config /home/airflow/.kube/config ENV KUBECONFIG=/home/airflow/.kube/config RUN chown airflow /home/airflow/.kube/config COPY increment.py $AIRFLOW_HOME/dags/increment.py USER airflow RUN pip install sh ``` values.yaml: ``` executor: KubernetesExecutor defaultAirflowRepository: 192.168.90.13:30500/airflow defaultAirflowTag: freezebug images: airflow: pullPolicy: Always flower: pullPolicy: Always pod_template: pullPolicy: Always logs: persistence: enabled: true size: 1Gi ``` ### What happened I had defined a DAG which uses `kubectl run` to run `bash -c 'sleep X'` for successively higher values of X: - 0 sec - 0.1 sec - 0.2 sec - ... - 2 sec Around 0.3 seconds, the tasks stopped completing: <img width="381" alt="stuck_eventually" src="https://user-images.githubusercontent.com/5834582/131287433-a8abaf1d-cfaa-4bcd-93ef-e754b20c41de.png";> ### What you expected to happen I expected the tasks to all complete successfully ### How to reproduce I made[ a repo](https://github.com/MatrixManAtYrService/bug_airflowfreeze) with the files that I used. See [commands.txt](https://github.com/MatrixManAtYrService/bug_airflowfreeze/blob/main/commands.txt) for the flow. Generally: - docker build an image with the problematic DAG, include kubectl and a kubeconfig file for a running cluster - push the image to a repository (I use https://github.com/twuni/docker-registry.helm to deploy one locally) - deploy Airflow in that image with the airflow helm chart - run the "increment" DAG - notice that it never completes ### Anything else If you use the Sequential Executor, it completes just fine ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
![https://user-images.githubusercontent.com/5834582/131287433-a8abaf1d-cfaa-4bcd-93ef-e754b20c41de.png"](https://user-images.githubusercontent.com/5834582/131287433-a8abaf1d-cfaa-4bcd-93ef-e754b20c41de.png"){kind=link}