potiuk commented on code in PR #41260:
URL: https://github.com/apache/airflow/pull/41260#discussion_r1704003495
##########
airflow/models/taskinstance.py:
##########
@@ -3892,9 +3892,12 @@ def schedule_downstream_tasks(self, session: Session =
NEW_SESSION, max_tis_per_
:meta: private
"""
- return TaskInstance._schedule_downstream_tasks(
- ti=self, session=session, max_tis_per_query=max_tis_per_query
- )
+ try:
+ return TaskInstance._schedule_downstream_tasks(
+ ti=self, session=session, max_tis_per_query=max_tis_per_query
+ )
+ except Exception:
+ self.log.exception("Error scheduling downstream tasks. Skipping
it")
Review Comment:
I **think** it's still useful to know that it's happening - the main thing
it currently struggles with is the exit code that is now 0. And it was quite
confusing because task succeeded, but the process failed.
I imagine this log will provide some useful information - for example if we
just silently discard it in https://github.com/apache/airflow/issues/39717 -
we would never have known that the problem is with serializing. Which **might**
be something theat the user can do something about - we have no idea, because
there might be varioue reasons for the exception.
I believe it is quite good to explicitly tell "something happen" - and while
yes - we do not know exactly what the user can do to avoid it, it will at least
give the user a chance to understand and fix it (or maybe submit an issue /
discussion here and get help.
I am generally very, very cautious about totally ignoring and silencing such
exceptions - because we have no idea what's going on. In this case we decided
that we won't do a confusing thing - because mini-scheduler is purely optional
we can just not fail the process, but having a log is still useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
