Thanks, Dan. We have not run into any issue. I was reading through this article <https://medium.com/handy-tech/airflow-tips-tricks-and-pitfalls-9ba53fba14eb>, where it talks about the scheduler restart, and also talks about Airbnb also does it, and got curious as it's easier to put this as a part of the process now than later.
~Manish On Thu, Jul 13, 2017 at 1:58 PM, Dan Davydov <[email protected] > wrote: > Airbnb switched to running the scheduler without restarts about a month > ago, and it is working well. The one issue you might hit is if you are > using a version of Celery with a bug (where workers will pull off tasks > even if they don't have space to run them), tasks can potentially be lost > once they are sent to your Celery backend. With restarts these tasks are > automatically resent. > > On Thu, Jul 13, 2017 at 1:38 PM, manish ranjan <[email protected]> > wrote: > > > Hi All, > > > > We have recently started to use airflow to make our data engineering more > > robust. > > 1. I wanted to know if there are good practices around scheduling a > daily/ > > bi-weekly/ weekly restart of the airflow server and scheduler? We are > using > > just the instance and not cluster. > > 2. Also wanted to know the troubles I may run into if someone is ready > to > > share based on experience? > > > > ~Manish > > >
