Although Airflow has the concept of task priority like Ash mentioned, it
does not pre-empt running tasks.
On Wed, Oct 9, 2019 at 12:42 AM Ash Berlin-Taylor wrote:
> There's already a concept called priority_weight on tasks
>
I just wanted to share this with everyone related to this topic. I have a
case where I need a scheduled dag run with following schedule:
Sun: 10 PM
Monday-Thursday: 8 AM, 11AM, 3:40 PM, 4:00 PM, 7:30 PM, 10PM
Friday: 8 AM, 11AM, 3:40 PM, 4:00 PM, 7:30 PM
There are most certainly a few ways to
Sure! Thank you very much!
Tomek
On Wed, Oct 9, 2019 at 9:51 AM Aizhamal Nurmamat kyzy
wrote:
> @Tomasz Urbaszek I got another ticket for
> the ApacheCon EU that you can use :) Are you still interested?
>
> On Fri, Oct 4, 2019 at 6:26 PM Jarek Potiuk
> wrote:
>
>> Yep. That's the only
@Tomasz Urbaszek I got another ticket for the
ApacheCon EU that you can use :) Are you still interested?
On Fri, Oct 4, 2019 at 6:26 PM Jarek Potiuk
wrote:
> Yep. That's the only Airflow talk :)
>
> J.
>
> On Fri, Oct 4, 2019 at 5:26 PM Driesprong, Fokko
> wrote:
>
> > I already got a ticket,
There's already a concept called priority_weight on tasks
http://airflow.apache.org/concepts.html?highlight=priority_weight#pools (the
doc about it is in relation to pools, but everything is run in a pool of
"default_pool" if not specified.)
Is that what you want?
On 9 October 2019 07:38:38
Hi,
Is there any discussion thread on adding priority to tasks and cost-based
optimization?
priority and pre-emption as an option to the user. If priority is
specified, scheduler has to schedule high priority tasks and if pre-emption
is true, it can pre-empt current running task which is of lower
