Romsik788 opened a new issue, #28941: URL: https://github.com/apache/airflow/issues/28941
### Apache Airflow version 2.5.0 ### What happened I am using this [docker-compose](https://airflow.apache.org/docs/apache-airflow/2.5.0/docker-compose.yaml) file to deploy Airflow locally on my machine. When I kill and restart the redis container, the worker Airflow reconnects (according to the logs), but after starting the tasks, they all become queued and run indefinitely. Last task:  I saw a solution that was adopted in Airflow 2.3.1, but it doesn't seem to work or I'm probably doing something wrong - #23690 ### What you think should happen instead After restarting redis, the tasks should run as before and not hang in the queue indefinitely. ### How to reproduce 1. Deploy Airflow using the official [docker-compose](https://airflow.apache.org/docs/apache-airflow/2.5.0/docker-compose.yaml) file. 2. Stop and start the redis container. 3. Run a task of some kind ### Operating System Ubuntu 22.04 LTS ### Versions of Apache Airflow Providers _No response_ ### Deployment Docker-Compose ### Deployment details _No response_ ### Anything else Part of logs, after I restarted the radish container. You can see that the worker lost the connection but then restores: ``` airflow-worker_1 | [2023-01-14 13:51:04,008: ERROR/MainProcess] consumer: Cannot connect to redis://redis:6379/0: Error -3 connecting to redis:6379. Temporary failure in name resolution.. airflow-worker_1 | Trying again in 8.00 seconds... (4/100) airflow-worker_1 | redis_1 | 1:M 14 Jan 2023 13:51:08.059 * monotonic clock: POSIX clock_gettime redis_1 | 1:M 14 Jan 2023 13:51:08.060 * Running mode=standalone, port=6379. redis_1 | 1:M 14 Jan 2023 13:51:08.060 # Server initialized redis_1 | 1:M 14 Jan 2023 13:51:08.060 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. redis_1 | 1:M 14 Jan 2023 13:51:08.060 * Loading RDB produced by version 7.0.7 redis_1 | 1:M 14 Jan 2023 13:51:08.060 * RDB age 95 seconds redis_1 | 1:M 14 Jan 2023 13:51:08.060 * RDB memory usage when created 1.42 Mb redis_1 | 1:M 14 Jan 2023 13:51:08.060 * Done loading RDB, keys loaded: 3, keys expired: 0. redis_1 | 1:M 14 Jan 2023 13:51:08.060 * DB loaded from disk: 0.000 seconds redis_1 | 1:M 14 Jan 2023 13:51:08.060 * Ready to accept connections airflow-worker_1 | [2023-01-14 13:51:12,021: INFO/MainProcess] Connected to redis://redis:6379/0 airflow-worker_1 | [2023-01-14 13:51:12,028: INFO/MainProcess] mingle: searching for neighbors airflow-worker_1 | [2023-01-14 13:51:13,037: INFO/MainProcess] mingle: all alone ``` ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
