brs-fcoo commented on issue #31544: URL: https://github.com/apache/airflow/issues/31544#issuecomment-2843050983
The issue isn't the general priority weight, that is set as desired, but a specific (not normal) situation might occur where we would like to have a task jump the queue (e.g. change the priority for that single task instance). The challenge for us, where the run button has been appreciated, is e.g. after serious downtime like a day or so, where we have to catch up with our forecasts (ocean and weather) and products. To give a specific example (real case): We have a big DAG that can have 3 simultaneous DAG runs (under normal circumstances there will not be more than one at a time). After a power outtage we are 6 DAG runs behind. As there a maybe 100GB data to download and process there a many tasks (thousands) to be run and we end up having a single task i the first DAG run queued for a long time as the same tasks in the two other runs of course have the same priority weight, which causes the fourth DAG run to be halted. He we have used the 'run' button to manually coerce a few tasks to run immediately so the DAG run can be finished, and the next one can start (downloading a lot of data). This is of course just 1 of about 30 DAGs which also needs to catch up. Just to be clear, everything is working as intended and our Airflow setup runs like a charm, but would just take longer if we didn't use the run button as a single task can block a whole DAG run from starting. So this is just an example where we have seen the benefit of being able to jump queue. Our setup is complex and could be designed differently, but I believe that no matter how we do similar situations would happen. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
