romanzdk commented on issue #33282: URL: https://github.com/apache/airflow/issues/33282#issuecomment-1675860636
- How many schedulers do you run? 3 - How many workers do you run? 1 - Which setup, Redis+Celery? Which executors? (SequentialExecutor) - Are you sure that no DB Backup especially no DB restore was made during the error cause? yes - Were any components of Scheduler, Worker, Redis Queue, Database restarted in the timeframe of the error? no - Is it possible to re-produce this? sometimes when I retry the dag instance it appears and sometimes no - Did it happen once or multiple times? every day since ~ start of the month - Would it be possible to have logs of all workers and schedulers in the time period shared? Would be a longer search for errors but might be the only final evidence. will try to get them - What is your [Scheduler] orphaned_tasks_check_interval and scheduler_zombie_task_threshold configuration? orphaned_tasks_check_interval = 300.0, scheduler_zombie_task_threshold = 300 - How long are the affected tasks running (start/end)? approximately 20mins - Do you have configured any specific [celery_broker_transport_options] visibility_timeout options? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
