Thanks Maxime. On Wed, Mar 8, 2017 at 10:18 AM, Maxime Beauchemin < [email protected]> wrote:
> It prioritizes before sending in the message queue. This assume that there > are tasks that cannot be scheduled because of dependency limitations as in > Airflow pools are full or max_concurrency is reached. Once the task is in > the message queue (Celery) Airflow has no more control over it. > > Max > > On Tue, Mar 7, 2017 at 7:01 PM, Amit Jain <[email protected]> wrote: > > > Please provide your input to this misconception. > > > > On Mar 6, 2017 11:46 PM, "Amit Jain" <[email protected]> wrote: > > > > > Hi All, > > > > > > I have a doubt related to task instance priority. When we specify the > > > priority_weight on the operator, does task priority re-ordering happens > > > between the eligible task instances at the time of Scheduler run or > does > > it > > > re-order queue in AMQP supported broker (RabbitMQ supports > priority_queue > > > 3.5.0 onwards)? > > > > > > I think first case is happening here. Here is code from the master > > branch. > > > > > > // jobs.py > > > def _execute_task_instances(....): > > > > > > priority_sorted_task_instances = sorted( > > > task_instances, key=lambda ti: (-ti.priority_weight, > > > ti.execution_date)) > > > > > > > > > > > > -- > > > Thanks, > > > Amit > > > > > >
