mitchelladam commented on issue #51569: URL: https://github.com/apache/airflow/issues/51569#issuecomment-3878241350
the issue here is probably related to: https://github.com/apache/airflow/pull/32572 what is happening is that when the scheduler runs its loop to pick up tasks from the database, it doesn't pick up enough tasks, leaving them queued for days. the solution is to set: scheduler.max_tis_per_query = 1024 (or 512 etc.) this will mean that tasks take 'longer' to go from queued to scheduled, because the scheduler loop picking up tasks to schedule takes longer. but at least they will make it out of being queued. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
