karenbraganz opened a new issue, #63240: URL: https://github.com/apache/airflow/issues/63240
### Apache Airflow version 3.1.7 ### If "Other Airflow 3 version" selected, which one? _No response_ ### What happened? If a task is deferred, the value of ti.start_date from the context dictionary reflects the timestamp when the task started re-running on a worker after deferral, not the actual start date of the task instance. For example, I ran a task using the DateTimeSensorAsync operator. It actually started at `2026-03-10 02:06:33` and immediately got deferred. ``` [2026-03-10 02:06:33] INFO - DAG bundles loaded: dags-folder [2026-03-10 02:06:33] INFO - Filling up the DagBag from /usr/local/airflow/dags/dts.py ... [2026-03-10 02:06:33] INFO - Pausing task as DEFERRED. dag_id=datetime_sensor_plus_3_minutes task_id=wait_3_minutes_after_start run_id=manual__2026-03-10T02:06:32.983547+00:00 ``` The trigger completed three minutes later at `2026-03-10 02:09:34`. ``` [2026-03-10 02:09:34] INFO - trigger completed name=datetime_sensor_plus_3_minutes/manual__2026-03-10T02:06:32.983547+00:00/wait_3_minutes_after_start/-1/1 (ID 3) ``` The task then re-ran on a worker at `2026-03-10 02:09:35`. ``` [2026-03-10 02:09:35] INFO - DAG bundles loaded: dags-folder [2026-03-10 02:09:35] INFO - Filling up the DagBag from /usr/local/airflow/dags/dts.py ``` When I printed `context["ti"].start_date` in my success callback, its value was `2026-03-10T02:09:35Z`. ``` [2026-03-10 02:09:35] INFO - • *Task started at*: 2026-03-10T02:09:35Z [2026-03-10 02:09:35] INFO - • *Task ended at*: 2026-03-10T02:09:35Z [2026-03-10 02:09:35] INFO - • *Task ran for*: 0 seconds! ``` Therefore, the start date is not getting calculated properly. The Details page shows the accurate start date. <img width="414" height="110" alt="Image" src="https://github.com/user-attachments/assets/35ffe8f3-4deb-4418-87a2-f283a8c0545e"; /> I have received reports of this happening with other deferrable operators too, so it is not unique to DateTimeSensorAsync. ### What you think should happen instead? The ti.start_date value should be the actual timestamp when the task began, not when it started running after deferral. ### How to reproduce Run this dag and observe the start date value printed by the callback. ``` from __future__ import annotations import datetime as dt from airflow.sdk import dag, task from airflow.providers.standard.sensors.date_time import DateTimeSensorAsync def print_success_callback(context): ti = context["ti"] ti_start_date = ti.start_date ti_end_date = ti.end_date run_duration_seconds = (ti_end_date - ti_start_date).seconds if run_duration_seconds < 60: duration = run_duration_seconds time_unit = "seconds" elif 60 <= run_duration_seconds < 3600: duration = round(run_duration_seconds / 60, 2) time_unit = "minutes" else: duration = round(run_duration_seconds / 3600, 2) time_unit = "hours" msg = ( f"• *Task started at*: {ti_start_date.strftime('%Y-%m-%dT%XZ')}\n" f"• *Task ended at*: {ti_end_date.strftime('%Y-%m-%dT%XZ')}\n" f"• *Task ran for*: {duration} {time_unit}!" ) print(msg) @dag( dag_id="datetime_sensor_plus_3_minutes", start_date=dt.datetime(2025, 1, 1), schedule="@daily", catchup=False, tags=["airflow3", "deferrable", "sensor"], ) def datetime_sensor_plus_3_minutes(): wait_3_minutes = DateTimeSensorAsync( task_id="wait_3_minutes_after_start", target_time="{{ dag_run.start_date + macros.timedelta(minutes=3) }}", on_success_callback=print_success_callback, ) @task def downstream(): print("3 minutes have passed since the DAG run started.") wait_3_minutes >> downstream() datetime_sensor_plus_3_minutes() ``` ### Operating System Debian GNU/Linux 12 (bookworm) ### Versions of Apache Airflow Providers _No response_ ### Deployment Astronomer ### Deployment details _No response_ ### Anything else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
