It prioritizes before sending in the message queue. This assume that there are tasks that cannot be scheduled because of dependency limitations as in Airflow pools are full or max_concurrency is reached. Once the task is in the message queue (Celery) Airflow has no more control over it.
Max On Tue, Mar 7, 2017 at 7:01 PM, Amit Jain <[email protected]> wrote: > Please provide your input to this misconception. > > On Mar 6, 2017 11:46 PM, "Amit Jain" <[email protected]> wrote: > > > Hi All, > > > > I have a doubt related to task instance priority. When we specify the > > priority_weight on the operator, does task priority re-ordering happens > > between the eligible task instances at the time of Scheduler run or does > it > > re-order queue in AMQP supported broker (RabbitMQ supports priority_queue > > 3.5.0 onwards)? > > > > I think first case is happening here. Here is code from the master > branch. > > > > // jobs.py > > def _execute_task_instances(....): > > > > priority_sorted_task_instances = sorted( > > task_instances, key=lambda ti: (-ti.priority_weight, > > ti.execution_date)) > > > > > > > > -- > > Thanks, > > Amit > > >
