dimberman opened a new issue #7911: Add data retention policy to Airflow
URL: https://github.com/apache/airflow/issues/7911
 
 
   **Description**
   
   Airflow's DB currently holds the entire history of all executions for all 
time. This is problematic as the DB grows. The UI starts to get slower, and the 
DB's disk usage grows. There is no bound to how large the DB will grow.
   
   It would be useful to add a feature in Airflow to do two things:
   
       Delete old data from the DB
       Mark some lower watermark, past which DAG executions are ignored
   
   For example, (2) would allow you to tell the scheduler "ignore all data 
prior to a year ago". And (1) would allow Airflow to delete all data prior to 
January 1, 2015.
   
   
   **Use case / motivation**
   
   
   **Related Issues**
   
   Copied from https://issues.apache.org/jira/browse/AIRFLOW-108
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


With regards,
Apache Git Services

Reply via email to