Adam Trump created AIRFLOW-5570:
-----------------------------------
Summary: Allow to store DAGs on GIT
Key: AIRFLOW-5570
URL: https://issues.apache.org/jira/browse/AIRFLOW-5570
Project: Apache Airflow
Issue Type: New Feature
Components: core, DAG, scheduler
Affects Versions: 1.10.5
Reporter: Adam Trump
Today it is possible to store the DAGs only on a specific folder, which is
configurable.
I'd like to suggest to allow storing the DAGs on GIT.
This feature comes from the fact that Airflow already have this functionallity
- under the KubernetesExecutor.
Thus, it shouldn't take much time moving it out of that executor, to the
Airflow scheduler.
When you run Airflow as a cluster on different machines, since all of the
components (webserver, scheduler and worker) needs access to the DAGs file,
there is always a need of accessable mount.
It can be very helpful if we could store the DAGs on GIT, and configure all
components to pull from that GIT repo.
Just as today, the scheduler should pull every X seconds from that repository.
When you want to add new DAGs, you simply need to push it to that repo.
It will also allow us to have some version control over DAGs, which could be
useful for keeping cleaner code and order.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
