vapiravfif commented on issue #19622: URL: https://github.com/apache/airflow/issues/19622#issuecomment-976280104
@halilduygulu Thank you for verifying this on newer version, haven't gotten to this yet. Ultimately, increasing only the `parallelism` value should solve this (in my case I bumped it to 1024, just to be sure). The origin of the issue seems to be in scheduler limiting the query of tasks in "scheduled" state and waiting to be queued by the amount of potentially available slots. But then some of the tasks can be still limited by dag concurrency and other constraints, resulting that tasks which are not limited not being picked up for a long time... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
