tardunge opened a new pull request, #63928:
URL: https://github.com/apache/airflow/pull/63928
Add a new `dag_bundle_config_path` setting that allows loading DAG bundle
configurations from JSON files in a directory, enabling hot-reloading without
restarting the DAG processor.
**Motivation:** In Kubernetes environments, bundle configurations need to
change frequently (e.g., via ConfigMaps). The current `dag_bundle_config_list`
requires static config in `airflow.cfg` and a processor restart. This feature
allows mounting a directory of JSON files and having the DAG processor
automatically detect and apply changes.
**Key changes:**
- `DagBundlesManager`: new `_parse_config_from_path()`,
`check_config_path_changes()`, and `get_bundle_path_safe()` methods
- `DagFileProcessorManager`: reuses a single `DagBundlesManager` instance;
detects config path changes in the refresh loop, cleaning up removed bundles
(processors, file stats, DAGs)
- `ParseImportError.full_file_path()`: graceful handling when bundles are
removed
- `config.yml`: new `dag_bundle_config_path` option under `[dag_processor]`
**JSON config format:**
```json
{
"name": "my-git-repo",
"classpath": "airflow.providers.git.bundles.git.GitDagBundle",
"kwargs": {
"tracking_ref": "main",
"git_conn_id": "my_git_conn"
}
}
```
**Behavior:**
- When `dag_bundle_config_path` is set, it takes precedence over
`dag_bundle_config_list`
- The DAG processor monitors the directory for JSON file additions,
removals, and modifications via mtime tracking
- On change: reloads config, syncs to DB, cleans up removed bundle
processors/DAGs, force-refreshes active bundles
- Invalid JSON files are logged and skipped; duplicate bundle names are
warned and skipped
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Opus 4.6)
Generated-by: Claude Code (Opus 4.6) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
