github-actions[bot] opened a new pull request, #76468: URL: https://github.com/apache/airflow/pull/76468
* Keep mapped tasks when clearing with run on latest version Clearing on the latest Dag version re-checks the run's task instances against that version. When the producer of a mapped task's input is cleared in the same request, its new attempt has no XCom yet, so the mapped length cannot be resolved and every expanded instance was marked removed. Nothing expands a removed index again, so the mapped task never re-ran and its downstream tasks ran as if it had finished. The same happened when a new version switched a task from literal to XCom mapping before the producer had run. The scheduler already trims or extends a mapped task once its upstreams have finished, so keeping the instances lets it settle the count from the producer's new value. * Test that the scheduler keeps cleared mapped tasks on Dag change The removal is not limited to clearing with "run on latest version". For a run without a bundle version, the scheduler checks the run's integrity whenever the Dag's structure changes, and before the fix that check removed the mapped instances of a cleared producer that had not run again yet. * Restore removed mapped tasks when their producer re-runs on latest version A mapped index removed earlier, by a shrink or because the task left the Dag, stayed removed after a clear on the latest version re-ran its producer: the unresolved-count branch skipped the restore block, and _revise_map_indexes_if_mapped never brings a removed index back. (cherry picked from commit cd3ae408a1a0570920407ebe3833fe7da4d666d6) Co-authored-by: Ephraim Anierobi <[email protected]> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
