github-actions[bot] opened a new pull request, #71008:
URL: https://github.com/apache/airflow/pull/71008
The dashboard counted each state separately with a limit of 1000, so any
busy state
showed "1000+" instead of a real number. On a large install almost every
state sat at
the cap, leaving the panel with no usable figures at all.
Count the window in a single scan instead. When it fits, every count is
exact. When it
does not, report what was read as a lower bound, rounded down, which stays
far closer
to the real volume than a fixed cap. Dag runs and task instances are judged
separately,
since a window often holds few enough dag runs to count exactly while
holding far too
many task instances.
Counting a bounded number of rows is also cheaper than the old per-state
limits, which
scanned the whole window for any state that could not fill its own limit
(postgres, 12M
task instances):
window before after
15min 15 ms 4 ms
24h 3,270 ms 45 ms
7d 20,752 ms 46 ms
MySQL and SQLite show the same pattern.
The Dags list keeps its previous capped counts and now owns that constant.
(cherry picked from commit b316afb44dc9686f5c39aab678a9a89cb21d53d2)
Co-authored-by: Jed Cunningham
<[email protected]>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
