halilduygulu edited a comment on issue #19622: URL: https://github.com/apache/airflow/issues/19622#issuecomment-975789786
Hi I am having the same case I think. airflow version 2.2.2 , just upgraded from 2.0.2 A huge dag running with concurrency = 10, has 435 tasks and there are other dags which their last tasks are in scheduled state forever now. I increased max_tis_per_query = 300 , it was 100, but did not see any improvement. Weird thing is if I ran another dag it runs fine until last task, then stuck, now I got 5 dags waiting in last step with plenty workers idle. scheduler logs are keep looping over 20 task instace names from big dag. my config ``` parallelism = 32 max_active_tasks_per_dag = 14 max_active_runs_per_dag = 10 max_tis_per_query = 300 ``` what can I do for a workaround now? edit: when I paused this huge dag, immediately other dags' task instances were running and then completed. update: increased max tis to 500, changed dag a bit, now 380 tasks. Still blocking other dags' last tasks. I think that I have to increase pool size very high too https://github.com/apache/airflow/pull/19747/files#diff-bde85feb359b12bdd358aed4106ef4fccbd8fa9915e16b9abb7502912a1c1ab3R254 update3: I increased parallelism and default_queue size and max_tis_per_query until scheduler started to pick up other dag tasks. I can see in db query limit number is increasing with these parameter values going up. Looking forward to get a perm solution to this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
