potiuk commented on issue #39532: URL: https://github.com/apache/airflow/issues/39532#issuecomment-2106396684
@j-stew -> It's basic design of Airflow tasks -namely idempotency of tasks. In Airflow, every run of the tasks is idempotent, which means that re-running the task is equivalent to running it anew. And Airflow assumes the previous run did not happen. This assumption has many consequences for backfill and DAG dependency scenarios. > Note: Deferrable operators not viable here due to database slowness If you see any slowness there, please open a new discussion describing what is the scenario you see and slowness you see (but please - unless you are sure this is an airflow issue, open a discussion, not issue). And make sure to explain what "slowness" is the problem . There should not be any extra slowness of deferrable operators (if they are correctly implemented)- especially comparing to rescheduling tasks (which also heavily depends on database). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
