fuatcakici commented on issue #34604:
URL: https://github.com/apache/airflow/issues/34604#issuecomment-1737017338

   Hello, thanks for picking this up! Upon further inspection, I have realised 
that I misidentified the issue in place, however, the issue exists nonetheless.
   
   I have written the following simple DAG in order to reproduce the issue and 
have triggered it manually twice:
   
   ```
   from airflow import DAG
   from airflow.operators.python import PythonOperator
   from airflow.models import DagRun
   from airflow.utils.state import TaskInstanceState
   
   import datetime as dt
   
   
   with DAG(
       dag_id="test",
       start_date=dt.datetime(2023, 9, 26),
   ) as dag:
       def simple_function():
           return "simple function"
   
   
       def error_function():
           dag_runs = DagRun.find(dag_id="test")
           if len(dag_runs) == 1:
               raise Exception("Intentional error raised.")
           else:
               previous_dag_run = dag_runs[0]
               previous_task_instance = 
previous_dag_run.get_task_instance("error_task")
               previous_task_instance.set_state(TaskInstanceState.SUCCESS)
   
               previous_dag_run.dag = dag
               previous_dag_run.update_state()
       
       tasks = {}
   
       simple_task = "simple_task"
       tasks[simple_task] = PythonOperator(
           task_id=simple_task,
           python_callable=simple_function
       )
   
       error_task = "error_task"
       tasks[error_task] = PythonOperator(
           task_id=error_task,
           python_callable=error_function
       )
   
       tasks[simple_task] >> tasks[error_task]
   ```
   
   So as you can see, in the first DAG run, the `simple_task` succeeds and the 
`error_task` fails, as shown:
   
   ![Screenshot 2023-09-26 at 16 22 
45](https://github.com/apache/airflow/assets/95378780/3e97f53e-320b-44a2-a75f-8653d26a904f)
   
   After triggering it for the second time, I was expecting the error in 
question, but the DAG simply ran the expected way:
   
   <img width="304" alt="Screenshot 2023-09-27 at 10 42 05" 
src="https://github.com/apache/airflow/assets/95378780/c83f44eb-b3f6-4bf1-a4aa-9bf33405be87";>
   
   After this, I inspected the issue further as I said before, and realised 
that the `update_state` function only fails when the DAG run state is SUCCESS. 
Hence, when I clear the `error_function` in the second DAG run, the task fails:
   
   <img width="300" alt="Screenshot 2023-09-27 at 10 47 18" 
src="https://github.com/apache/airflow/assets/95378780/03e6dd3a-1615-4d3b-8cf7-68493a719251";>
   
   with the following stack trace:
   
   ```
   [2023-09-27, 08:47:07 UTC] {{taskinstance.py:1936}} ERROR - Task failed with 
exception
   Traceback (most recent call last):
     File 
"/home/airflow/airflow_env/lib/python3.10/site-packages/airflow/operators/python.py",
 line 192, in execute
       return_value = self.execute_callable()
     File 
"/home/airflow/airflow_env/lib/python3.10/site-packages/airflow/operators/python.py",
 line 209, in execute_callable
       return self.python_callable(*self.op_args, **self.op_kwargs)
     File "/usr/local/airflow/dags/data_flows/dags/test/dag.py", line 28, in 
error_function
       previous_dag_run.update_state()
     File 
"/home/airflow/airflow_env/lib/python3.10/site-packages/airflow/utils/session.py",
 line 77, in wrapper
       return func(*args, session=session, **kwargs)
     File 
"/home/airflow/airflow_env/lib/python3.10/site-packages/airflow/models/dagrun.py",
 line 724, in update_state
       self._emit_duration_stats_for_finished_state()
     File 
"/home/airflow/airflow_env/lib/python3.10/site-packages/airflow/models/dagrun.py",
 line 964, in _emit_duration_stats_for_finished_state
       Stats.timing(f"dagrun.duration.{self.state.value}.{self.dag_id}", 
**timer_params)
   AttributeError: 'str' object has no attribute 'value'
   ```
   
   At this point, I wondered if this was somehow because the DAG run state was 
already right, i.e., would be changed from SUCCESS to SUCCESS since all the 
task instances in it were also successful. However, the `update_state` function 
also fails when one of the DAG run state is SUCCESS, but one of the task 
instances within it is FAILED. You can see this behaviour by manually setting 
the `simple_task` in the first DAG run to FAILED, and clearing the `error_task` 
in the second DAG run:
   
   <img width="303" alt="Screenshot 2023-09-27 at 10 55 58" 
src="https://github.com/apache/airflow/assets/95378780/ddec61f9-ac31-4e17-9eb6-88403f7ab55d";>
   
   The stack trace is the same.
   
   Hence, replacing `self.state.value` with `self.state` will not solve the 
issue, as it will likely create another issue for the DAG runs in the FAILED 
state. Thus, my intuition is that somewhere in the code, the `self.state` for 
successful DAG runs is returned as only `'success'`, but for failed DAG runs it 
is returned as `<DagRunState.FAILED: 'failed'>`, but I cannot figure out where.
   
   Hope this helps, and let me know if there is anything else I can do!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to