potiuk commented on code in PR #39890:
URL: https://github.com/apache/airflow/pull/39890#discussion_r1623281077
##########
airflow/jobs/local_task_job_runner.py:
##########
@@ -303,6 +306,14 @@ def heartbeat_callback(self, session: Session =
NEW_SESSION) -> None:
if dagrun_timeout and execution_time > dagrun_timeout:
self.log.warning("DagRun timed out after %s.",
execution_time)
+ # If process still runs after being marked as success, let it run
until configured overtime
+ # if there are configured listeners
+ if (
+ ti.state == TaskInstanceState.SUCCESS
+ and self._overtime < conf.getint("core",
"task_listener_timeout")
+ and get_listener_manager().has_listeners
Review Comment:
Do we want to limit it to only case where we have listeners? I think very
similar scenario might happen when there is a "mini-scheduler" running after
task completes. I **think** that case would also benefit from eager heartbeat
killing such task. Maybe then we should turn it into a generic "task_timeout"
not "task_listener_timeout" and kill two birds with the same stone?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
