ephraimbuddy opened a new pull request, #27506: URL: https://github.com/apache/airflow/pull/27506
We have a case where the mini scheduler tries to expand a mapped task even when the downstream tasks are not yet done. The mini scheduler extracts a partial subset of a dag and in the process, some upstream tasks are dropped. If the task happens to be a mapped task, the expansion will fail since it needs the upstream output to make the expansion. When the expansion fails, the task is marked as `upstream_failed`. This leads to other downstream tasks being marked as upstream failed. The solution was to ignore this error and not mark the mapped task as upstream_failed when the expansion fails and the dag is a partial subset -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
