malthe opened a new pull request #21316: URL: https://github.com/apache/airflow/pull/21316
This fixes an issue that can occur when a task is deferred and the resulting trigger completes before the executor is notified that the task was completed, whereby the rescheduling of the task fails since the executor refuses to enqueue a task which is already running. The executor's tracking of running tasks is always lagging behind the task instance state but this is typically not a problem because tasks are not rescheduled fast enough for this to be a problem. But with deferred tasks and triggering logic, this picture changes since a trigger condition can in some cases be met right away – for example, checking whether a certain external system has a given state (i.e., "sensoring"). An alternative solution is to for example sleep for a small amount of time in the triggerer before changing the task state back to `SCHEDULED` – perhaps in the form of a minimum triggering duration to allow the executor to register that the initial task has completed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
