kaxil opened a new pull request, #72909: URL: https://github.com/apache/airflow/pull/72909
## Problem A Dag author pushes a commit and has no way to tell whether Airflow has picked it up. The workflow today is: commit, push, then reload the Dags page until the import errors or the file details change. Where Dag code viewing is disabled there is not even that, and the author is left guessing the bundle refresh interval. Nothing in the UI or the REST API exposes Dag bundle state, though `dag_bundle` has recorded the version and last refresh since Airflow 3.0. ## Solution `GET /api/v2/dagBundles` lists each bundle with the version Airflow currently holds, when a Dag processor last refreshed it, how many Dag import errors it has, its owning team, and whether it is still configured. A **Dag Bundles** page under *Browse* renders that and refreshes itself while open. For a Git bundle `version` is the commit SHA of the tracking ref, comparable directly against what was deployed, which also makes "wait until my commit is live" scriptable from a deployment pipeline. `last_refreshed` advances on every successful refresh even when the version did not change, so it doubles as "a processor is alive and looking at this bundle". A non-zero `import_error_count` beside a version that just moved means the code was fetched but a Dag in it failed to import. Read-only on purpose: no way to create, delete, enable or disable a bundle, so no new role and no new writable configuration surface. That keeps it clear of the objection on #61645, which is about runtime *mutation* of bundle configuration. ## Authorization, the decision I would most like a second opinion on A bundle is visible to a caller who can read at least one Dag recorded against it. The route carries `requires_access_dag(method="GET")` and rows are filtered by `PermittedDagBundleFilter`, the existing `PermittedDagFilter` pattern applied to `dag_bundle` through the Dags inside it. `import_error_count` is counted on the same terms as `GET /importErrors`, and `bundle_url` is withheld from callers who may not read Dag versions. I deliberately did not add an `AccessView.DAG_BUNDLES`. It is the house style for admin-ish read views, but it needs a FAB `RESOURCE_DAG_BUNDLE` seeded into roles, and seeding that admin- or op-only locks out the Dag authors this exists for. I also rejected `is_authorized_dag(method="GET", details=DagDetails(id=None, team_name=...))`. FAB reads that as "may this user list Dags", which `_is_authorized_list_dags` answers True for anyone holding read on a single Dag, and FAB ignores `team_name` outright, so one Dag's worth of access would have exposed every bundle's repository URL. Deriving from readable Dags behaves identically across FAB, SimpleAuthManager and Keycloak. #72715 faces the same question and asked it without an answer, so I am raising it rather than treating either choice as settled. ## Known limitations - It cannot tell you a refresh is *failing*: a raising `bundle.refresh()` is caught and logged, leaving `last_refreshed` at its last success with no error marker, so "failing for an hour" looks like "not due yet". Documented, with the two checks that need no refresh interval. A `last_refresh_error` column would fix it properly and is follow-up work. - A new version does not mean the new Dags are running; the bundle is refreshed before its Dags are parsed. - A bundle from which no Dag has ever parsed successfully is not listed at all. Screenshots and a short demo of the page updating live are being added to this description. Related: #61645, #72715, #60616. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
