atul-astronomer opened a new issue, #49047: URL: https://github.com/apache/airflow/issues/49047
### Apache Airflow version main (development) ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? When user is reading dags from GitBundle and backfilling previous version dagruns then task state is in removed state. https://github.com/user-attachments/assets/dde14240-78bd-4acb-ba17-9bfb9d270c34 ### What you think should happen instead? _No response_ ### How to reproduce 1. Setup init.sh file to read dags from GitBundle ```python export AIRFLOW_CONN_GIT_DEFAULT='{ "conn_type": "git", "host": "https://github.com/atul-astronomer/dag_version_test.git" }' export AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST='[ { "name": "dag_version_test", "classpath": "airflow.providers.git.bundles.git.GitDagBundle", "kwargs": { "subdir": "dags1", "tracking_ref": "main", "refresh_interval": 5 } }]' ``` 2. Use below dag code to create multiple dagruns on past dates. ```python from datetime import datetime from airflow.sdk import Asset from airflow.sdk.definitions.asset.metadata import Metadata # from airflow.sdk.metadata import Metadata from airflow.decorators import task from pendulum import today from airflow.sdk import DAG outlet = Asset('asset_outlet') with DAG( dag_id="test_asset_event_producer", start_date=today('UTC').add(days=-5), schedule='@daily', tags=["asset", "AIP-74"], is_paused_upon_creation=False, catchup=True ) as dag: @task(outlets=[outlet]) def asset_with_extra_by_yield(): yield Metadata(outlet, {"hi": "bye"}) asset_with_extra_by_yield() ``` 3. Once multiple dagruns are available on version 1, change the version using below code. ```python from datetime import datetime from airflow.sdk import Asset from airflow.sdk.definitions.asset.metadata import Metadata # from airflow.sdk.metadata import Metadata from airflow.decorators import task from pendulum import today from airflow.sdk import DAG outlet = Asset('asset_outlet') with DAG( dag_id="test_asset_event_producer", start_date=today('UTC').add(days=-5), schedule='@daily', tags=["asset", "AIP-74"], is_paused_upon_creation=False, catchup=True ) as dag: @task(outlets=[outlet]) def asset_with_extra_by_yield2(): yield Metadata(outlet, {"hi": "bye2"}) asset_with_extra_by_yield2() ``` 4. Create a dagrun on version 2. 5. Now create backfill on previous dates to override version 1 dagruns. 6. Notice backfill dagruns have succeeded but task is in removed state. ### Operating System Linux ### Versions of Apache Airflow Providers _No response_ ### Deployment Other ### Deployment details _No response_ ### Anything else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
