Just make sure to archive based on start_date and not execution_date to allow backfills.
Max On Mon, Aug 28, 2017 at 3:20 PM, Alex Guziel <[email protected] > wrote: > Here at Airbnb we delete old "completed" task instances. > > On Mon, Aug 28, 2017 at 3:01 PM, David Capwell <[email protected]> wrote: > > > We are on 1.8.0 and have a monitor DAG that monitors the health of > Airflow > > and Celery every minute. This has been running for awhile now and at 26k > > dag runs. We see that the UI for this DAG is multiple seconds slower (6-7 > > second) than any other DAG. > > > > My question is, what do people do about managing history as it grows over > > time? Do people delete history after N or so days? > > > > Thanks for your time reading this email > > >
