seanghaeli opened a new issue, #77061: URL: https://github.com/apache/airflow/issues/77061
`TriggerRunnerSupervisor.run_once` (`airflow-core/src/airflow/jobs/triggerer_job_runner.py`) waits for the trigger runner with `self._service_subprocess(1)`, which returns as soon as any of the runner's sockets has data, and then does a full pass of database work: `load_triggers` (`Trigger.assign_unassigned` and `Trigger.ids_for_triggerer`), `clean_unused` and the heartbeat check. The runner's log socket is in the same selector, so every line a trigger logs starts another pass. Requests from the runner, like connection and variable lookups, do the same. Measured on main (a81cfb7ac98) in Breeze with Postgres, the real triggerer job and one asset watcher that logs N lines a second (mean of 2 runs): | Trigger logs | Loop passes/s | SQL statements/s | Row-locking statements/s | |---|---:|---:|---:| | nothing | 1.0 | 7.5 | 4 | | 10 lines/s | 11.0 | 77.5 | 44 | | 50 lines/s | 51.0 | 357.6 | 204 | | 200 lines/s | 107.6 | 753.9 | 430.5 | Every pass is the same 7 statements, 4 of them row-locking (the three claim queries and the `clean_unused` select), so a triggerer's database load grows with how much its triggers log, not with how many triggers it runs. Suggestion: keep reading the runner's sockets as data arrives, but run the database part of the loop on a timer, about once a second, which is how often the runner syncs its state anyway. Done when: a triggerer's database passes stay at about one a second however much its triggers log, while log lines and runner requests are still handled as they arrive, with a test. Related to the triggerer performance work in #76209: syncing the runner more often would wake this loop the same way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
