ephraimbuddy commented on a change in pull request #17819:
URL: https://github.com/apache/airflow/pull/17819#discussion_r703413781
##########
File path: airflow/jobs/scheduler_job.py
##########
@@ -596,13 +597,24 @@ def _process_executor_events(self, session: Session =
None) -> int:
)
self.log.error(msg, ti, state, ti.state, info)
+ try:
+ get_dag(self.subdir, ti.dag_id)
Review comment:
When there's an error in parsing the dag, maybe due to resource
constraints, the task callback is not run even though it's sent to the
processor. The task gets stuck in queued.
But when there's db operational error while trying to execute a task, it
results in this error but the task retries.
One other thing we can do is add a message to inform the user that if
there's a resource constraint that the task can get stuck. Then remove parsing
of dags
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
