Re: Airflow meetup in Bangalore (India)

Sumit, Is this a good time to create a local Airflow meet-up group in Bangalore? We currently have 4 meet-ups : https://cwiki.apache.org/confluence/display/AIRFLOW/Meetups -s On Tue, Mar 27, 2018 at 3:01 AM, Sumit Maheshwari wrote: > Hi folks, > > Please fill out the small

Re: Awesome list of resources around Apache Airflow

Awesome. Thanks for the share, Max! - Jin On Wed, Mar 28, 2018 at 9:40 PM, Ace Haidrey wrote: > Thanks for the share! > > Sent from my iPhone > > > On Mar 28, 2018, at 9:30 PM, Tao Feng wrote: > > > > Thanks Max for sharing the link. This is great :)

Re: RBAC Update

Great work Joy. This is awesome! I am interested in helping out the per dag level access. Just created a ticket to check(AIRFLOW-2267). Let me know if you have any suggestions. I will share my proposal once I am ready. On Fri, Mar 23, 2018 at 6:45 PM, Joy Gao wrote: > Hey guys!

Re: Awesome list of resources around Apache Airflow

Thanks for the share! Sent from my iPhone > On Mar 28, 2018, at 9:30 PM, Tao Feng wrote: > > Thanks Max for sharing the link. This is great :) > > -Tao > > On Wed, Mar 28, 2018 at 9:21 PM, Maxime Beauchemin < > maximebeauche...@gmail.com> wrote: > >> I was pleasantly

Re: Awesome list of resources around Apache Airflow

This is awesome. Thanks for sharing the link. Regards, Ananth.P, On 28 March 2018 at 21:30, Tao Feng wrote: > Thanks Max for sharing the link. This is great :) > > -Tao > > On Wed, Mar 28, 2018 at 9:21 PM, Maxime Beauchemin < > maximebeauche...@gmail.com> wrote: > > >

Re: Awesome list of resources around Apache Airflow

Thanks Max for sharing the link. This is great :) -Tao On Wed, Mar 28, 2018 at 9:21 PM, Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > I was pleasantly surprised to stumble upon this recently: > https://github.com/jghoman/awesome-apache-airflow > > Please contribute anything that you

Awesome list of resources around Apache Airflow

I was pleasantly surprised to stumble upon this recently: https://github.com/jghoman/awesome-apache-airflow Please contribute anything that you think is missing from the list! Max

Re: Airflow Scalability with Local Executor

The LocalExecutor is great for running small numbers of DAGs/tasks, but it is more of a starter executor meant to made Airflow work out of the box. I would recommend switching to a different executor like the CeleryExecutor. You are certainly right that there is room for reducing the memory

Airflow Scalability with Local Executor

Hi All, We have a use case to support 1000 concurrent DAGs. These dags would have have couple of Http task which would be submitting jobs to external services. Each DAG could run for couple of hours. HTTP tasks are periodically checking(with sleep 20) the job status. We tried running 1000 such

Podling Report Reminder - April 2018

Dear podling, This email was sent by an automated system on behalf of the Apache Incubator PMC. It is an initial reminder to give you plenty of time to prepare your quarterly board report. The board meeting is scheduled for Wed, 18 April 2018, 10:30 am PDT. The report for your podling will form