Dask just landed in master. So no Celery is the most used option to scale-out.
Always interested in what you are running into, but please be prepared to provide a lot of info on your setup. - Boke > On 13 Feb 2017, at 17:01, EKC (Erik Cederstrand) <[email protected]> wrote: > > Hello all, > > > I'm investigating why some of our DAGs are not being scheduled properly ( ran > into https://issues.apache.org/jira/browse/AIRFLOW-342, among other things). > Coupled with comments on this list, I'm getting the impression that Celery is > a second-class citizen and core developers are mainly using Dask. Is this > correct? > > > If Dask support is simply more mature and more likely to have issues > responded to, I'll consider migrating our installation. > > > Thanks, > > Erik
