antonio-mello-ai commented on issue #62033: URL: https://github.com/apache/airflow/issues/62033#issuecomment-4048643487
Hi @pierrejeambrun, I'd like to work on this. After reading the code, the main bottleneck is `useGridTISummaries` being instantiated per-run in `TaskInstancesColumn`, generating N requests per refresh cycle. The 5x `slowRefreshMultiplier` for non-selected runs helps but doesn't address the core issue. **Proposed approach (incremental, frontend-only):** 1. **Skip refetch for runs in final states** (`success`, `failed`, `upstream_failed`) — their TI summaries won't change, so `refetchInterval: false`. This alone should eliminate most requests when the majority of displayed runs are completed. 2. **Reduce grid/structure refresh frequency** — DAG structure rarely changes mid-session. Could increase its interval or make it conditional on DAG version changes. 3. **Simple backoff for long-running runs** — if a TI summary response hasn't changed across consecutive fetches, progressively slow down the polling for that run. I intentionally left out batching (new backend endpoint) and WebSocket/SSE as they'd be larger changes. Happy to tackle those in a follow-up if you think the scope is right. Does this approach make sense, or would you prefer a different direction? Please let me know before I start coding. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
