[
https://issues.apache.org/jira/browse/AIRFLOW-3104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16631784#comment-16631784
]
ASF GitHub Bot commented on AIRFLOW-3104:
-----------------------------------------
ashb closed pull request #3939: [AIRFLOW-3104] Add .airflowignore info into doc
URL: https://github.com/apache/incubator-airflow/pull/3939
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/models.py b/airflow/models.py
index 6154ca64f4..6ea4638a51 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -522,10 +522,12 @@ def collect_dags(
Given a file path or a folder, this method looks for python modules,
imports them and adds them to the dagbag collection.
- Note that if a .airflowignore file is found while processing,
- the directory, it will behaves much like a .gitignore does,
+ Note that if a ``.airflowignore`` file is found while processing
+ the directory, it will behave much like a ``.gitignore``,
ignoring files that match any of the regex patterns specified
- in the file. **Note**: The patterns in .airflowignore are treated as
+ in the file.
+
+ **Note**: The patterns in .airflowignore are treated as
un-anchored regexes, not shell-like glob patterns.
"""
start_dttm = timezone.utcnow()
diff --git a/docs/concepts.rst b/docs/concepts.rst
index 7316477225..a30a7026bc 100644
--- a/docs/concepts.rst
+++ b/docs/concepts.rst
@@ -863,3 +863,32 @@ do the same, but then it is more to use a virtualenv and
pip.
to be available on the system if a module needs those. In other words only
pure python modules can be packaged.
+
+.airflowignore
+''''''''''''''
+
+A ``.airflowignore`` file specifies the directories or files in ``DAG_FOLDER``
+that Airflow should intentionally ignore. Each line in ``.airflowignore``
+specifies a regular expression pattern, and directories or files whose names
+(not DAG id) match any of the patterns would be ignored (under the hood,
+``re.findall()`` is used to match the pattern). Overall it works like a
+``.gitignore`` file.
+
+``.airflowignore`` file should be put in your ``DAG_FOLDER``.
+For example, you can prepare a ``.airflowignore`` file with contents
+
+.. code::
+
+ project_a
+ tenant_[\d]
+
+
+Then files like "project_a_dag_1.py", "TESTING_project_a.py", "tenant_1.py",
+"project_a/dag_1.py", and "tenant_1/dag_1.py" in your ``DAG_FOLDER`` would be
ignored
+(If a directory's name matches any of the patterns, this directory and all its
subfolders
+would not be scanned by Airflow at all. This improves efficiency of DAG
finding).
+
+The scope of a ``.airflowignore`` file is the directory it is in plus all its
subfolders.
+You can also prepare ``.airflowignore`` file for a subfolder in ``DAG_FOLDER``
and it
+would only be applicable for that subfolder.
+
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> .airflowignore feature is not mentioned at all in documentation
> ---------------------------------------------------------------
>
> Key: AIRFLOW-3104
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3104
> Project: Apache Airflow
> Issue Type: Improvement
> Components: Documentation
> Reporter: Xiaodong DENG
> Assignee: Xiaodong DENG
> Priority: Minor
> Fix For: 2.0.0
>
>
> This is a nice feature, but not mentioned at all in anywhere of the
> documentation.
> Relevant information can only be found in source code comment.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
