dstandish commented on code in PR #45318:
URL: https://github.com/apache/airflow/pull/45318#discussion_r1904483207
##########
airflow/config_templates/config.yml:
##########
@@ -2676,30 +2676,38 @@ dag_bundles:
description: |
Configuration for the DAG bundles. This allows Airflow to load DAGs from
different sources.
- Airflow will consume all options added to this section. Below you will see
only the default,
- ``dags_folder``. The option name is the bundle name and the value is a
json object with the following
- keys:
-
- * classpath: The classpath of the bundle class
- * kwargs: The keyword arguments to pass to the bundle class
- * refresh_interval: The interval in seconds to refresh the bundle from its
source.
+ options:
+ backends:
+ description: |
+ List of backend configs. Must supply name, classpath, and kwargs for
each backend.
- For example, to add a new bundle named ``hello`` to my Airflow instance,
add the following to your
- airflow.cfg (this is just an example, the classpath and kwargs are not
real):
+ By default, ``refresh_interval`` is set to ``[scheduler]
dag_dir_list_interval``, but that can
+ also be overridden in kwargs if desired.
- .. code-block:: ini
+ The default is the dags folder dag bundle.
- [dag_bundles]
- hello: {classpath: "airflow.some.classpath", kwargs: {"hello":
"world"}, refresh_interval: 60}
- options:
- dags_folder:
- description: |
- This is the default DAG bundle that loads DAGs from the traditional
``[core] dags_folder``.
- By default, ``refresh_interval`` is set to ``[scheduler]
dag_dir_list_interval``, but that can be
- overridden here if desired.
- Parsing DAGs from the DAG folder can be disabled by setting this
option to an empty string.
- version_added: ~
+ Note: As shown below, you can split your json config over multiple
lines by indenting.
+ See configparser documentation for an example:
+
https://docs.python.org/3/library/configparser.html#supported-ini-file-structure.
+ version_added: 3.0.0
type: string
- example: ~
- default: '{{"classpath":
"airflow.dag_processing.bundles.dagfolder.DagsFolderDagBundle",
- "kwargs": {{}}}}'
+ example: >
+ [
+ {
+ "name": "my-git-repo",
+ "classpath": "airflow.dag_processing.bundles.git.GitDagBundle",
+ "kwargs": {
+ "subdir": "dags",
+ "repo_url": "[email protected]:example.com/my-dags.git",
+ "tracking_ref": "main",
+ "refresh_interval": 0
+ }
+ ]
+ default: >
+ [
+ {{
+ "name": "dags-folder",
Review Comment:
basically, to just help distinguish from "dags_folder" the path, and the the
airlfow setting that has existed since beginning of time.
question for you. are we going to deprecate `dags_folder` as a setting? i
forget if you said that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
