ashb commented on code in PR #47678:
URL: https://github.com/apache/airflow/pull/47678#discussion_r1991737924
##########
airflow/jobs/scheduler_job_runner.py:
##########
@@ -1061,14 +1061,16 @@ def _run_scheduler_loop(self) -> None:
with create_session() as session:
# This will schedule for as many executors as possible.
num_queued_tis = self._do_scheduling(session)
+ # Don't keep any objects alive -- we've possibly just
looked at 500+ ORM objects!
+ session.expunge_all()
- # Heartbeat all executors, even if they're not receiving
new tasks this loop. It will be
- # either a no-op, or they will check-in on currently
running tasks and send out new
- # events to be processed below.
- for executor in self.job.executors:
- executor.heartbeat()
+ # Heartbeat all executors, even if they're not receiving new
tasks this loop. It will be
+ # either a no-op, or they will check-in on currently running
tasks and send out new
+ # events to be processed below.
+ for executor in self.job.executors:
+ executor.heartbeat()
Review Comment:
This is so that executor.heartbeat is not called during an active
transaction -- given we don't pass a session on, if a specific executor wants
to do DB actions it will need to create it's own session anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
