jscheffl commented on issue #47518: URL: https://github.com/apache/airflow/issues/47518#issuecomment-2707518070
Error trace 2 I could capture from scheduler POD stdout:
```
2025-03-07 21:14:29 [debug ] Workload process exited [supervisor]
exit_code=<Negsignal.SIGKILL: -9>
2025-03-07 21:14:29 [info ] Process exited [supervisor]
exit_code=<Negsignal.SIGKILL: -9> pid=153 signal=SIGKILL
[2025-03-07T21:14:29.228+0000] {local_executor.py:101} ERROR - uhoh
Traceback (most recent call last):
File "/opt/airflow/airflow/executors/local_executor.py", line 97, in
_run_worker
_execute_work(log, workload)
File "/opt/airflow/airflow/executors/local_executor.py", line 118, in
_execute_work
supervise(
File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/supervisor.py",
line 1067, in supervise
process = ActivitySubprocess.start(
File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/supervisor.py",
line 665, in start
proc._on_child_started(ti=what, dag_rel_path=dag_rel_path,
bundle_info=bundle_info)
File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/supervisor.py",
line 675, in _on_child_started
ti_context = self.client.task_instances.start(ti.id, self.pid,
start_date)
File "/opt/airflow/task-sdk/src/airflow/sdk/api/client.py", line 134, in
start
resp = self.client.patch(f"task-instances/{id}/run",
content=body.model_dump_json())
File "/home/airflow/.local/lib/python3.9/site-packages/httpx/_client.py",
line 1219, in patch
return self.request(
File
"/home/airflow/.local/lib/python3.9/site-packages/tenacity/__init__.py", line
336, in wrapped_f
return copy(f, *args, **kw)
File
"/home/airflow/.local/lib/python3.9/site-packages/tenacity/__init__.py", line
475, in __call__
do = self.iter(retry_state=retry_state)
File
"/home/airflow/.local/lib/python3.9/site-packages/tenacity/__init__.py", line
376, in iter
result = action(retry_state)
File
"/home/airflow/.local/lib/python3.9/site-packages/tenacity/__init__.py", line
398, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 439, in
result
return self.__get_result()
File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 391, in
__get_result
raise self._exception
File
"/home/airflow/.local/lib/python3.9/site-packages/tenacity/__init__.py", line
478, in __call__
result = fn(*args, **kwargs)
File "/opt/airflow/task-sdk/src/airflow/sdk/api/client.py", line 442, in
request
return super().request(*args, **kwargs)
File "/home/airflow/.local/lib/python3.9/site-packages/httpx/_client.py",
line 827, in request
return self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/home/airflow/.local/lib/python3.9/site-packages/httpx/_client.py",
line 914, in send
response = self._send_handling_auth(
File "/home/airflow/.local/lib/python3.9/site-packages/httpx/_client.py",
line 942, in _send_handling_auth
response = self._send_handling_redirects(
File "/home/airflow/.local/lib/python3.9/site-packages/httpx/_client.py",
line 999, in _send_handling_redirects
raise exc
File "/home/airflow/.local/lib/python3.9/site-packages/httpx/_client.py",
line 982, in _send_handling_redirects
hook(response)
File "/opt/airflow/task-sdk/src/airflow/sdk/api/client.py", line 99, in
raise_on_4xx_5xx
return get_json_error(response) or response.raise_for_status()
File "/opt/airflow/task-sdk/src/airflow/sdk/api/client.py", line 95, in
get_json_error
raise err
airflow.sdk.api.client.ServerResponseError: Server returned error
```
...after this the scheduler POD was just hanging and un-responsive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
