Gollum999 opened a new issue, #25744: URL: https://github.com/apache/airflow/issues/25744
### Apache Airflow version 2.3.3 ### What happened If you try to `backfill` a DAG that uses [repeated dynamic task mapping](https://airflow.apache.org/docs/apache-airflow/2.3.0/concepts/dynamic-task-mapping.html#repeated-mapping), some of the repeated mapped tasks will be marked as "removed". Any tasks downstream of the removed tasks will refuse to start:  Or, if there is nothing downstream of those tasks, the DAG Run will claim it was successful, even though some of its tasks never ran:  ### What you think should happen instead All mapped tasks should run as normal during a backfill. ### How to reproduce ``` #!/usr/bin/env python3 import datetime import itertools import logging import string from airflow.decorators import dag, task logger = logging.getLogger(__name__) TASK_COUNT = 25 @dag( schedule_interval='@daily', start_date=datetime.datetime(2022, 8, 16), default_args={ 'retries': 0, }, ) def test_backfill_removed(): @task def get_tasks(): return list(itertools.islice(itertools.cycle(string.ascii_letters), TASK_COUNT)) @task def t1(arg): logger.info(f'{arg=}') return arg t1.expand(arg=t1.expand(arg=get_tasks())) # t1.expand(arg=t1.expand(arg=t1.expand(arg=get_tasks()))) dag = test_backfill_removed() if __name__ == '__main__': dag.cli() ``` ``` airflow dags backfill test_backfill_removed -s 2022-08-01 -e 2022-08-05 ``` ### Operating System CentOS Stream 8 ### Versions of Apache Airflow Providers None ### Deployment Other ### Deployment details Self-hosted + standalone ### Anything else I've reproduced this with SQLite + `SequentialExecutor` (standalone), Postgres + `LocalExecutor` (standalone), and Postgres + `CeleryExecutor` (production deployment, self-hosted). I don't see any errors or warnings in the logs. In fact, even with `logging_level = DEBUG`, I don't see *any* log lines that reference the tasks that are listed as "removed". This error is sometimes inconsistent to reproduce - if you are unlucky, you might instead get the deadlock mentioned in #25730. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
