Might have a look at this, too: https://github.com/teamclairvoyant/airflow-maintenance-dags
I haven't used it, but it seems to have a DB cleaning script. On Mon, Aug 28, 2017 at 3:43 PM, Maxime Beauchemin < [email protected]> wrote: > Just make sure to archive based on start_date and not execution_date to > allow backfills. > > Max > > On Mon, Aug 28, 2017 at 3:20 PM, Alex Guziel <[email protected]. > invalid > > wrote: > > > Here at Airbnb we delete old "completed" task instances. > > > > On Mon, Aug 28, 2017 at 3:01 PM, David Capwell <[email protected]> > wrote: > > > > > We are on 1.8.0 and have a monitor DAG that monitors the health of > > Airflow > > > and Celery every minute. This has been running for awhile now and at > 26k > > > dag runs. We see that the UI for this DAG is multiple seconds slower > (6-7 > > > second) than any other DAG. > > > > > > My question is, what do people do about managing history as it grows > over > > > time? Do people delete history after N or so days? > > > > > > Thanks for your time reading this email > > > > > >
