Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-22 Thread Darren Weber
> > >>> > J. > > > > > >>> > > > > > > >>> > > > > > > >>> > On Tue, Jan 14, 2020 at 12:18 AM Darren Weber < > > > > > >>> dweber.consult...@gmail.com > > > > > >>> &

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-20 Thread Kaxil Naik
;>> > > > > > > >>> > wrote: > > > > >>> > > > > > >>> > > +1 for keeping it and fixing tests > > > > >>> > > > > > > >>> > > PS, I also noticed the skipped test

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-20 Thread Darren Weber
> how the test setup needs to work, I might also discover how to > > > enable > > > >>> > tests > > > >>> > > for the non-async executor. No promises, just noting that I'm > > aware > > > >>> of

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-20 Thread Jarek Potiuk
ecutor. No promises, just noting that I'm > aware > > >>> of it > > >>> > > too. > > >>> > > > > >>> > > On Mon, Jan 13, 2020 at 8:06 AM Jarek Potiuk < > > >>> jarek.pot...@polidea.com> > > >>> >

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-20 Thread Darren Weber
e > >>> executed and > >>> > > > summarized as XFail tests and we will have to deal with them at > >>> some > >>> > > point. > >>> > > > > >>> > > > I think we will have to decide if we w

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-19 Thread Jarek Potiuk
ts and we will have to deal with them at >>> some >>> > > point. >>> > > > >>> > > > I think we will have to decide if we want to keep it or not, and >>> either >>> > > > remove both tests and executor or fix the

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-18 Thread Jarek Potiuk
> > >> > > > J. >> > > > >> > > > On Mon, Jan 13, 2020 at 4:40 PM Shaw, Damian P. < >> > > > damian.sha...@credit-suisse.com> wrote: >> > > > >> > > > > FYI I used Dash instead of Local Executor w

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-17 Thread Jarek Potiuk
gt; > > > > was a great way to make sure the Executor and Scheduler weren’t > tied > > to > > > > > each other with no difficulty in set-up. But once I actually > started > > > > > deploying to multiple boxes I needed queue names pretty quickly. So

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-17 Thread Daniel Imberman
loying to multiple boxes I needed queue names pretty quickly. So > not > > > > going to say it's needed but for me it was a helpful stepping stone. > > > > > > > > > > > > -Original Message- > > > > From: Ash Berlin-Taylor > >

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-17 Thread Jarek Potiuk
r with no difficulty in set-up. But once I actually started > > > > deploying to multiple boxes I needed queue names pretty quickly. So > not > > > > going to say it's needed but for me it was a helpful stepping stone. > > > > > > > > > > >

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-17 Thread Daniel Imberman
oing to say it's needed but for me it was a helpful stepping stone. > > > > > > > > > -----Original Message- > > > From: Ash Berlin-Taylor > > > Sent: Sunday, January 12, 2020 17:38 > > > To: dev@airflow.apache.org > > > Cc: dev@

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-17 Thread Jarek Potiuk
started > > > deploying to multiple boxes I needed queue names pretty quickly. So not > > > going to say it's needed but for me it was a helpful stepping stone. > > > > > > > > > -----Original Message- > > > From: Ash Berlin-Taylor > > &

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-13 Thread Darren Weber
mes pretty quickly. So not > > going to say it's needed but for me it was a helpful stepping stone. > > > > > > -Original Message- > > From: Ash Berlin-Taylor > > Sent: Sunday, January 12, 2020 17:38 > > To: dev@airflow.apache.org > > Cc: dev

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-12 Thread Ash Berlin-Taylor
ail when I try to run the tests. I tried to look for any reference in > devlist archives but I couldn't find anything about it. > > Can someone shed some light on this? Should we remove Dask executor > completely from Airflow 2.0 ? Or should we fix the tests/executor ? Has it >

Remove Dask Executor in Airflow 2.0 ?

2020-01-12 Thread Jarek Potiuk
all tests are skipped. And they fail when I try to run the tests. I tried to look for any reference in devlist archives but I couldn't find anything about it. Can someone shed some light on this? Should we remove Dask executor completely from Airflow 2.0 ? Or should we fix the tests/executor ? Has it been