johnny0120 commented on issue #16163: URL: https://github.com/apache/airflow/issues/16163#issuecomment-1231041796
Having the same problem with Airflow 2.2.5 + celery + redis. The default visibility_timeout is actually 21600, 6 hours (not None from the airflow.cfg) , which means if the worker cannot pick up the task successfully, the task_instance will be stuck in the queued state for 6 hours and not visible to other workers. I remember when clearing tasks in this situation won't help either, may need to mark task failed first and then clear it. If I change visibility_timeout to a smaller number, say 300, 5 minutes, then due the default settings of in `airflow.config_templates.default_celery.DEFAULT_CELERY_CONFIG`, `'task_acks_late': True`, the task_instance will be killed by other workers if it keeps running after 5 minutes, complaining about the task is still in the running state. (Or I'm not sure if that is the same case when ack is lost from the worker) Can we just change the default setting of `task_acks_late` to False? Will it cause any side effects? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
