argibbs commented on code in PR #29933:
URL: https://github.com/apache/airflow/pull/29933#discussion_r1130578134
##########
airflow/sensors/external_task.py:
##########
@@ -130,29 +142,31 @@ def __init__(
):
super().__init__(**kwargs)
self.allowed_states = list(allowed_states) if allowed_states else
[State.SUCCESS]
+ self.skipped_states = list(skipped_states) if skipped_states else []
self.failed_states = list(failed_states) if failed_states else []
- total_states = set(self.allowed_states + self.failed_states)
+ total_states = set(self.allowed_states + self.skipped_states +
self.failed_states)
- if set(self.failed_states).intersection(set(self.allowed_states)):
+ if len(total_states) != len(self.allowed_states + self.skipped_states
+ self.failed_states):
Review Comment:
(FWIW though, in our install, by far and away the most expensive part of the
external task sensor poke method is the db query for the state of the target
... depending on the state of the rest of the system it can be 10s of seconds -
we have some big dags and the scheduler poll puts fairly massive load on the db
- someone appears to have reported this as #25448. Not that that's any excuse
for inefficient code mind you!)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
