tmccall8829 opened a new issue, #30869:
URL: https://github.com/apache/airflow/issues/30869
### Description
Hey all! I'm working on building an internal tool for visualizing the
dependency graph and gantt chart for all of our cross-dag dependencies in our
entire airflow instance. I was planning on building the graph on top of data
retrieved from the Airflow v2 REST API, specifically the `/dags` and the
`/dags/{dag_id}/tasks` endpoints.
I was hoping to be able to see in the API response for TriggerDagRunOperator
tasks some info about the DAG that task triggers, but I don't see anything in
the response for DAGs or DAG tasks that I can use for this purpose. I only see
within-DAG task dependencies, nothing about cross-dag dependencies. Is there an
endpoint in the API (or, alternatively, maybe something stored in the metadata
db) I could use for that purpose?
It seems like maybe the way the API serves up DAG and task info is very
confined to within-DAG information and not cross-dag info... but please correct
me if I'm wrong or misunderstanding how the API should be used 🙂
### Use case/motivation
For example, let's say I have a really simple setup with two DAGs: A and B.
A runs first, and at the end of the run has a TriggerDagRunOperator that
triggers DAG B by name. So the simple dependency graph there would be something
like:
`A_t1 <- A_t2 <- (B_t*)` (i.e., DAG B and all associated tasks only run when
triggered by A_t2).
I was thinking that it'd be nice, e.g., to be able to GET `/dags/B` and have
it include something like:
```
{
...
"dag_id": "B",
"upstream_dag_trigger": ["A"],
...
}
```
(Or, alternatively, request info about A_t2 and have it list something like
"triggered_downstream_dag" name).
I guess theoretically I could construct this myself by reading in A's source
code via REST API and read the actual task info for the TriggerDagRunOperator
task as defined in the DAG itself, but that feels cumbersome for an airflow
setup with, say, hundreds of DAGs. Do you all have any ideas there? Am I sort
of just asking for something that can't really be done?
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
