[
https://issues.apache.org/jira/browse/AIRFLOW-5523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942070#comment-16942070
]
Dan Davydov commented on AIRFLOW-5523:
--------------------------------------
My concern with that would be that this could filter out new dags, especially
ones that the user only intends to trigger manually. Maybe we can have some
heuristic where if the DAG was added X days ago and still has not run we hide
it, but I'm not sure that it's necessary.
I think eventually along your lines of thinking it would be nice to have a
notification sent out about DAGs that have not run in a long time for
clean-up/scheduler resources purposes, or maybe even going a step further and
being able to automatically expiring and hiding these DAGS, although that might
be better suited as a plugin or something like that.
> DAGs never get cleaned up from webservers if DAG file is removed from the
> scheduler
> -----------------------------------------------------------------------------------
>
> Key: AIRFLOW-5523
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5523
> Project: Apache Airflow
> Issue Type: Bug
> Components: webserver
> Affects Versions: 1.10.5
> Reporter: Dan Davydov
> Priority: Major
> Attachments: Screen Shot 2019-09-19 at 4.39.24 PM.png
>
>
> DAGs are never removed from the DB if the DAG file used to generate the DAGs
> is removed. This causes the webserver to accumulate broken dag rows (see
> attached screenshot).
> The scheduler should delete all DAGs from the DAG table with files that no
> longer exist when it scans the DAGs folder for files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
