Lee-W commented on PR #71072: URL: https://github.com/apache/airflow/pull/71072#issuecomment-6845939073
> Would it work to move the count and the audit write into a periodic callback instead? Yes, `_report_partition_cap_backlog` is now a periodic callback (60 seconds, registered with `non_fatal=True`) that opens its own session and does the per-Dag `GROUP BY`, the warning and the audit rows. The scheduler tick keeps only the row-locked, `LIMIT`-capped fetch, so a steady-state backlog no longer costs an extra count per loop. The separate audit session and the empty/short-fetch handling are removed. The per-Dag edge trigger is unchanged. The reported set is only updated after the session commits, so a failed write leaves the Dags unreported and the next run retries. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
