m1racoli commented on code in PR #56976:
URL: https://github.com/apache/airflow/pull/56976#discussion_r2449480084
##########
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/triggers/pod.py:
##########
@@ -183,6 +188,11 @@ async def run(self) -> AsyncIterator[TriggerEvent]:
)
return
except Exception as e:
+ self.log.exception(
Review Comment:
> Spent the resumed normal operator already log it too from the info in the
event?
If `trigger_reentry` fails, then a single log line of the TriggerEvent is
the only thing we have. It does include the stacktrace but only on a single
logged line.
I'd lean towards logging the stack trace where it happens and not relying on
downstream code to print `stack_trace` out, which it may never do if it fails
before that step.
This is what is currently available on a single line in the scenario I
described (where `trigger_reentry` fails before printing the stack trace).
```
[2025-10-17T06:40:27.475+0000] {triggerer_job_runner.py:631} INFO - Trigger
<omitted>/scheduled__2025-10-16T05:00:00+00:00/<omitted>/-1/1 (ID 5973774)
fired: TriggerEvent<{'name': '<omitted>', 'namespace': '<omitted>', 'status':
'error', 'message': '(404)\nReason: Not Found\nHTTP response headers:
<CIMultiDictProxy(\'Audit-Id\': \'<omitted>\', \'Cache-Control\': \'no-cache,
private\', \'Content-Type\': \'application/json\',
\'X-Kubernetes-Pf-Flowschema-Uid\': \'<omitted>\',
\'X-Kubernetes-Pf-Prioritylevel-Uid\': \'<omitted>\', \'Date\': \'Fri, 17 Oct
2025 06:40:27 GMT\', \'Content-Length\': \'294\')>\nHTTP response body:
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods
\\"<omitted>\\" not
found","reason":"NotFound","details":{"name":"<omitted>","kind":"pods"},"code":404}\n\n',
'stack_trace': 'Traceback (most recent call last):\n File
"/usr/local/lib/python3.12/site-packages/airflow/providers/cncf/kubernetes/triggers/pod.py",
line 170, in run\n
event = await self._wait_for_container_completion()\n
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/site-packages/airflow/providers/cncf/kubernetes/triggers/pod.py",
line 236, in _wait_for_container_completion\n pod = await self._get_pod()\n
^^^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/site-packages/tenacity/asyncio/__init__.py", line
189, in async_wrapped\n return await copy(fn, *args, **kwargs)\n
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/site-packages/tenacity/asyncio/__init__.py", line
111, in __call__\n do = await self.iter(retry_state=retry_state)\n
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/site-packages/tenacity/asyncio/__init__.py", line
153, in iter\n result = await action(retry_state)\n
^^^^^^^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/site-packages/tenacity/_utils.py", line 99, in
inner\n retur
n call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/site-packages/tenacity/__init__.py", line 418, in
exc_check\n raise retry_exc.reraise()\n ^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/site-packages/tenacity/__init__.py", line 185, in
reraise\n raise self.last_attempt.result()\n
^^^^^^^^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result\n
return self.__get_result()\n ^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in
__get_result\n raise self._exception\n File
"/usr/local/lib/python3.12/site-packages/tenacity/asyncio/__init__.py", line
114, in __call__\n result = await fn(*args, **kwargs)\n
^^^^^^^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/site-packages/airflow/providers/cncf/kubernetes/triggers/pod.py",
line 276, in _get_pod\n pod = await self.hook.get_pod(name=s
elf.pod_name, namespace=self.pod_namespace)\n
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n
File
"/usr/local/lib/python3.12/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py",
line 853, in get_pod\n pod: V1Pod = await v1_api.read_namespaced_pod(\n
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/site-packages/kubernetes_asyncio/client/api_client.py",
line 192, in __call_api\n raise e\n File
"/usr/local/lib/python3.12/site-packages/kubernetes_asyncio/client/api_client.py",
line 185, in __call_api\n response_data = await self.request(\n
^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/site-packages/kubernetes_asyncio/client/rest.py",
line 212, in GET\n return (await self.request("GET", url,\n
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File
"/usr/local/lib/python3.12/site-packages/kubernetes_asyncio/client/rest.py",
line 206, in request\n raise ApiExc
eption(http_resp=r)\nkubernetes_asyncio.client.exceptions.ApiException:
(404)\nReason: Not Found\nHTTP response headers:
<CIMultiDictProxy(\'Audit-Id\': \'<omitted>\', \'Cache-Control\': \'no-cache,
private\', \'Content-Type\': \'application/json\',
\'X-Kubernetes-Pf-Flowschema-Uid\': \'<omitted>\',
\'X-Kubernetes-Pf-Prioritylevel-Uid\': \'<omitted>\', \'Date\': \'Fri, 17 Oct
2025 06:40:27 GMT\', \'Content-Length\': \'294\')>\nHTTP response body:
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods
\\"<omitted>\\" not
found","reason":"NotFound","details":{"name":"<omitted>","kind":"pods"},"code":404}\n\n\n'}>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
