[
https://issues.apache.org/jira/browse/AIRFLOW-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17236084#comment-17236084
]
Ash Berlin-Taylor commented on AIRFLOW-2221:
--------------------------------------------
Which isn't to say this won't come, we just haven't planned it yet.
Discussion about this is happening in
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-5+Remote+DAG+Fetcher
> Fill up DagBag from remote locations
> ------------------------------------
>
> Key: AIRFLOW-2221
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2221
> Project: Apache Airflow
> Issue Type: New Feature
> Components: configuration, core
> Affects Versions: 2.0.0
> Reporter: Diogo Franco
> Assignee: Chao-Han Tsai
> Priority: Major
> Fix For: 2.0.0
>
>
> The ability to fill up the DagBag from remote locations (HDFS, S3...) seems
> to be deemed useful, e.g. facilitating deployment processes.
> This JIRA is to propose an implementation of a *DagFetcher* abstraction on
> the DagBag, where the collect_dags method can delegate the walking to a
> *FileSystemDagFetcher*, *GitRepoDagFetcher*, *S3DagFetcher*,
> *HDFSDagFetcher*, *GCSDagFetcher*, *ArtifactoryDagFetcher* or even
> *TarballInS3DagFetcher*.
> This was briefly discussed in [this mailing list
> thread|https://lists.apache.org/thread.html/03ddcd3a42b7fd6e3dad9711e8adea37fc00391f6053762f73af5b6a@%3Cdev.airflow.apache.org%3E]
> I'm happy to start work on this and provide an initial implementation for
> review.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
