Thanks, will take a look at this project On Aug 29, 2017 10:35 AM, "Chris Riccomini" <[email protected]> wrote:
> Might have a look at this, too: > > https://github.com/teamclairvoyant/airflow-maintenance-dags > > I haven't used it, but it seems to have a DB cleaning script. > > On Mon, Aug 28, 2017 at 3:43 PM, Maxime Beauchemin < > [email protected]> wrote: > > > Just make sure to archive based on start_date and not execution_date to > > allow backfills. > > > > Max > > > > On Mon, Aug 28, 2017 at 3:20 PM, Alex Guziel <[email protected]. > > invalid > > > wrote: > > > > > Here at Airbnb we delete old "completed" task instances. > > > > > > On Mon, Aug 28, 2017 at 3:01 PM, David Capwell <[email protected]> > > wrote: > > > > > > > We are on 1.8.0 and have a monitor DAG that monitors the health of > > > Airflow > > > > and Celery every minute. This has been running for awhile now and at > > 26k > > > > dag runs. We see that the UI for this DAG is multiple seconds slower > > (6-7 > > > > second) than any other DAG. > > > > > > > > My question is, what do people do about managing history as it grows > > over > > > > time? Do people delete history after N or so days? > > > > > > > > Thanks for your time reading this email > > > > > > > > > >
