GitHub user gopidesupavan added a comment to the discussion: [Bug]
`standard.operators.python` fails the task randomly
> The full scheduler return error:
>
> ```
> [2025-08-11T11:46:06.659+0000] {dagrun.py:1039} INFO - Found span_status
> 'ended', while updating state for dag_run
> 'manual__2025-08-11T11:20:19.498701+00:00'
> 2025-08-11 11:46:06 [debug ] Received message from task runner
> [supervisor] msg=SucceedTask(state='success',
> end_date=datetime.datetime(2025, 8, 11, 11, 46, 6, 660306,
> tzinfo=datetime.timezone.utc), task_outlets=[], outlet_events=[],
> rendered_map_index=None, type='SucceedTask')
> [2025-08-11T11:46:06.719+0000] {_client.py:1026} INFO - HTTP Request: PATCH
> http://localhost:8080/execution/task-instances/019898ef-0cfd-7ba2-a315-cb5caf28eb36/state
> "HTTP/1.1 204 No Content"
> [2025-08-11T11:46:07.714+0000] {dagrun.py:1176} INFO - Marking run <DagRun
> Insights_IPv6_Adoption @ 2025-08-11 11:20:18.505000+00:00:
> manual__2025-08-11T11:20:19.498701+00:00, state:running, queued_at:
> 2025-08-11 11:41:00.039941+00:00. run_type: manual> successful
> [2025-08-11T11:46:07.715+0000] {dagrun.py:1226} INFO - DagRun Finished:
> dag_id=Insights_IPv6_Adoption, logical_date=2025-08-11 11:20:18.505000+00:00,
> run_id=manual__2025-08-11T11:20:19.498701+00:00, run_start_date=2025-08-11
> 11:41:00.180309+00:00, run_end_date=2025-08-11 11:46:07.715110+00:00,
> run_duration=307.534801, state=success, run_type=manual,
> data_interval_start=2025-08-11 11:20:18.505000+00:00,
> data_interval_end=2025-08-11 11:20:18.505000+00:00,
> 2025-08-11 11:46:11 [info ] Task finished [supervisor]
> duration=310.4626057599671 exit_code=0 final_state=success
> [2025-08-11T11:46:12.779+0000] {scheduler_job_runner.py:811} INFO - Received
> executor event with state success for task instance
> TaskInstanceKey(dag_id='Insights_IPv6_Adoption',
> task_id='download_cloudflare_ipv6_data',
> run_id='manual__2025-08-11T11:20:19.498701+00:00', try_number=5, map_index=-1)
> [2025-08-11T11:46:12.783+0000] {scheduler_job_runner.py:853} INFO -
> TaskInstance Finished: dag_id=Insights_IPv6_Adoption,
> task_id=download_cloudflare_ipv6_data,
> run_id=manual__2025-08-11T11:20:19.498701+00:00, map_index=-1,
> run_start_date=2025-08-11 11:41:01.324087+00:00, run_end_date=2025-08-11
> 11:46:06.660306+00:00, run_duration=310.336, state=success,
> executor=LocalExecutor(parallelism=64), executor_state=success, try_number=5,
> max_tries=4, pool=default_pool, queue=default, priority_weight=1,
> operator=_PythonDecoratedOperator, queued_dttm=2025-08-11
> 11:41:01.254731+00:00, scheduled_dttm=2025-08-11
> 11:41:01.224247+00:00,queued_by_job_id=7, pid=1415904
> [2025-08-11T11:47:30.971+0000] {dagrun.py:1039} INFO - Found span_status
> 'ended', while updating state for dag_run
> 'manual__2025-08-11T11:20:19.498701+00:00'
> [2025-08-11T11:47:30.999+0000] {scheduler_job_runner.py:451} INFO - 1 tasks
> up for execution:
> <TaskInstance: Insights_IPv6_Adoption.download_cloudflare_ipv6_data
> manual__2025-08-11T11:20:19.498701+00:00 [scheduled]>
> [2025-08-11T11:47:30.999+0000] {scheduler_job_runner.py:523} INFO - DAG
> Insights_IPv6_Adoption has 0/32 running and queued tasks
> [2025-08-11T11:47:30.999+0000] {scheduler_job_runner.py:662} INFO - Setting
> the following tasks to queued state:
> <TaskInstance: Insights_IPv6_Adoption.download_cloudflare_ipv6_data
> manual__2025-08-11T11:20:19.498701+00:00 [scheduled]>
> [2025-08-11T11:47:31.000+0000] {scheduler_job_runner.py:768} INFO - Trying to
> enqueue tasks: [<TaskInstance:
> Insights_IPv6_Adoption.download_cloudflare_ipv6_data
> manual__2025-08-11T11:20:19.498701+00:00 [scheduled]>] for executor:
> LocalExecutor(parallelism=64)
> 2025-08-11 11:47:31 [info ] Secrets backends loaded for worker
> [supervisor] backend_classes=['EnvironmentVariablesBackend'] count=1
> [2025-08-11T11:47:31.038+0000] {_client.py:1026} INFO - HTTP Request: PATCH
> http://localhost:8080/execution/task-instances/019898f5-02a4-73c2-9e98-baf7be52acee/run
> "HTTP/1.1 409 Conflict"
> 2025-08-11 11:47:31 [warning ] Server error
> [airflow.sdk.api.client] detail={'detail': {'reason': 'invalid_state',
> 'message': 'TI was not in a state where it could be marked as running',
> 'previous_state': 'scheduled'}}
> 2025-08-11 11:47:31 [info ] Process exited [supervisor]
> exit_code=<Negsignal.SIGKILL: -9> pid=1419080 signal_sent=SIGKILL
>
>
> [2025-08-11T11:47:31.060+0000] {local_executor.py:96} ERROR - uhoh
> Traceback (most recent call last):
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/airflow/executors/local_executor.py",
> line 92, in _run_worker
> _execute_work(log, workload)
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/airflow/executors/local_executor.py",
> line 120, in _execute_work
> supervise(
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/airflow/sdk/execution_time/supervisor.py",
> line 1793, in supervise
> process = ActivitySubprocess.start(
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/airflow/sdk/execution_time/supervisor.py",
> line 854, in start
> proc._on_child_started(ti=what, dag_rel_path=dag_rel_path,
> bundle_info=bundle_info)
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/airflow/sdk/execution_time/supervisor.py",
> line 865, in _on_child_started
> ti_context = self.client.task_instances.start(ti.id, self.pid, start_date)
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/airflow/sdk/api/client.py",
> line 151, in start
> resp = self.client.patch(f"task-instances/{id}/run",
> content=body.model_dump_json())
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/httpx/_client.py",
> line 1219, in patch
> return self.request(
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/tenacity/__init__.py",
> line 338, in wrapped_f
> return copy(f, *args, **kw)
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/tenacity/__init__.py",
> line 477, in __call__
> do = self.iter(retry_state=retry_state)
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/tenacity/__init__.py",
> line 378, in iter
> result = action(retry_state)
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/tenacity/__init__.py",
> line 400, in <lambda>
> self._add_action_func(lambda rs: rs.outcome.result())
> File "/usr/lib64/python3.9/concurrent/futures/_base.py", line 439, in result
> return self.__get_result()
> File "/usr/lib64/python3.9/concurrent/futures/_base.py", line 391, in
> __get_result
> raise self._exception
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/tenacity/__init__.py",
> line 480, in __call__
> result = fn(*args, **kwargs)
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/airflow/sdk/api/client.py",
> line 714, in request
> return super().request(*args, **kwargs)
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/httpx/_client.py",
> line 827, in request
> return self.send(request, auth=auth, follow_redirects=follow_redirects)
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/httpx/_client.py",
> line 914, in send
> response = self._send_handling_auth(
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/httpx/_client.py",
> line 942, in _send_handling_auth
> response = self._send_handling_redirects(
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/httpx/_client.py",
> line 999, in _send_handling_redirects
> raise exc
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/httpx/_client.py",
> line 982, in _send_handling_redirects
> hook(response)
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/airflow/sdk/api/client.py",
> line 116, in raise_on_4xx_5xx
> return get_json_error(response) or response.raise_for_status()
> File
> "/home/zifengma/Airflow/.venv/lib64/python3.9/site-packages/airflow/sdk/api/client.py",
> line 112, in get_json_error
> raise err
> airflow.sdk.api.client.ServerResponseError: Server returned error
> ```
looks like the default value will not be considered when there is no
`PythonVirtualenvOperator.cache_key`
will check later today. thanks for logs
GitHub link:
https://github.com/apache/airflow/discussions/54534#discussioncomment-14113664
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
