shakeelansari63 opened a new issue, #40888:
URL: https://github.com/apache/airflow/issues/40888
### Apache Airflow version
Other Airflow 2 version (please specify below)
### If "Other Airflow 2 version" selected, which one?
2.9.2
### What happened?
Hello,
After upgrading to Airflow `2.9.2` we have started seeing a strange issue
with our Airflow tasks. Sometime if the task fails, the executor pod is
deleted. But somehow Airflow does not get notified of the failure causing the
task to continue running.
After some time, scheduler finds it as a daemon task and kills the task. I
don't know if this is airflow issue or Kubernetes Provider's issue. But there
seem to be some drop in communication between these two.
But this has been occurring very frequently since `2.9.2` upgrade.
In some rare instances, I have also seen this behavior after successful
completion of task. Though task complete successfully, Airflow is not notified
of task state. Causing the task being killed later by Scheduler. And task
status changes to failed.
### What you think should happen instead?
When the task fails and pod is killed, Airflow should get notified
appropriately. And task should reflect correct status.
### How to reproduce
I am not sure how to re-produce it. This is not something happening
everytime. It happens to some job. And on re-run the job completes correctly.
### Operating System
Ubuntu Linux (on AKS)
### Versions of Apache Airflow Providers
```
apache-airflow==2.9.2
apache-airflow-provides-cnfc-kubernetes==8.3.0
apache-airflow-provides-celery==3.7.1
apache-airflow-provides-ssh==3.11.1
apache-airflow-provides-ftp==3.9.1
apache-airflow-provides-sftp==4.9.0
apache-airflow-provides-common-sql==1.14.0
apache-airflow-provides-redis==3.6.0
apache-airflow-provides-jdbc==4.2.2
apache-airflow-provides-odbc==4.4.1
apache-airflow-provides-apache-spark==4.8.1
```
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
We have deployed it AKS
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
