meel-ayush opened a new issue, #76552:
URL: https://github.com/apache/airflow/issues/76552
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
3.4.0rc3 (and main)
### What happened and how to reproduce it?
### Description
When `bundle.refresh()` fails in the DAG processor manager (such as a git
network timeout, GitHub/GitLab rate limiting, or expired credentials),
`DagFileProcessorManager._refresh_dag_bundles()` catches the exception and logs
it to daemon logs, but records nothing in the metadata database.
Exactly 0 rows are written to `ParseImportError` or `DagWarning`. Because of
this, the Airflow Web UI displays 0 import errors and reports the environment
as healthy, even though DAG synchronization from the bundle has stopped
completely.
### Affected Code
In `airflow-core/src/airflow/dag_processing/manager.py:934-939`:
```python
try:
bundle.refresh()
any_refreshed = True
except Exception:
self.log.exception("Error refreshing bundle %s", bundle.name)
continue
```
### Steps to Reproduce
1. Configure a `GitDagBundle` pointing to a remote repository.
2. Trigger a connection failure (`ConnectionResetError`) or authentication
failure during `bundle.refresh()`.
3. The DAG processor logs `Error refreshing bundle ...` to stderr and
continues its loop.
4. Check the metadata DB: 0 rows in `import_error` (`ParseImportError`) and
0 rows in `dag_warning` (`DagWarning`).
5. The UI shows 0 import errors while the bundle remains out of date.
### Expected Behavior
When a bundle fails to refresh, a `ParseImportError` or `DagWarning` (e.g.
`DagWarningType.BUNDLE_REFRESH_FAILED`) should be recorded in the database for
that bundle so operators can see the failure directly on the UI dashboard.
### Proposed Fix
In `airflow-core/src/airflow/dag_processing/manager.py`, catch the exception
in `_refresh_dag_bundles` and persist a warning or import error row to the
database.
I have verified this behavior with a local test and am working on a PR with
regression tests to address this.
### What you think should happen instead?
_No response_
### Operating System
Kali GNU/Linux
### Deployment
None
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]