g-dailey opened a new issue, #57776:
URL: https://github.com/apache/airflow/issues/57776
### Apache Airflow version
3.1.1
### If "Other Airflow 2/3 version" selected, which one?
_No response_
### What happened?
While testing Airflow 3.1.1, I observed moderate performance degradation in
the Grid View when working with a large DAG that has around 180–200 tasks and
30+ dagruns.
After refreshing the Grid View several times or triggering new DAG runs, the
page loads very slowly and eventually becomes unresponsive. The issue becomes
apparent once the number of dagruns exceeds roughly 25.
The issue appears to stem from multiple areas:
• Backend inefficiency: TaskInstance summaries are fetched per
dagrun, leading to heavy and repetitive processing.
• Frontend inefficiency: The Grid View performs full data reloads
for all visible dagruns on every refresh.
• These behaviors are consistent with scaling limitations
previously reported in
[apache/airflow#50928](https://github.com/apache/airflow/issues/50928).
Addressing this likely requires coordinated changes to both backend batching
and frontend rendering logic.
Possible Improvements
• Implement batched TaskInstance summary fetching to avoid
redundant per-dagrun queries.
• Introduce pagination or virtualization for dagruns in the Grid
View.
• Optimize frontend rendering to reduce full data reprocessing.
### What you think should happen instead?
Expected Behavior
• Grid View should remain responsive even for DAGs with hundreds
of tasks and many historical dagruns.
• Memory usage should remain stable regardless of dagrun count or
refresh frequency.
• Triggering additional DAG runs should not cause UI or API lag.
### How to reproduce
Steps to Reproduce
1. Create or use a DAG with approximately 180–200 tasks.
2. Allow the DAG to accumulate 30+ dagruns.
3. Open the Grid View and refresh the page multiple times, or
trigger additional DAG runs a few times.
4. Observe increasing page load times and memory consumption.
### Operating System
Operating System: Debian GNU/Linux 12 (bookworm) (via Astro Runtime 3.1-3
container)
### Versions of Apache Airflow Providers
_No response_
### Deployment
Astronomer
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
