dstandish commented on issue #6370: AIRFLOW-5701: Don't clear xcom explicitly before execution URL: https://github.com/apache/airflow/pull/6370#issuecomment-547273710 What motivated this PR was the need to persist state through reschedules, in the service of creating an async operator. I think we can narrow the scope of this change to meet this need without causing these undesired consequences. Why not _only_ not clear at start when the task is resuming after a reschedule. There is a state `up_for_reschedule` which seems promising but it appears to be quite shortlived, i.e. it does not last until the task resumes. Perhaps an async operator, or a sensor in `reschedule` mode should take on a "hibernate"-like state when waiting for reschedule time. Something not occupying worker slot, but still considered part of the same task instance and same attempt, so that idempotence is still respected. When merely being reanimated following a hibernate, xcom is not purged; when retrying after a failure, xcom is purged.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
