jjournet commented on issue #24309:
URL: https://github.com/apache/airflow/issues/24309#issuecomment-1171576419
hello,
same issue at the end of a task, airflow 2.3.0
```
[2022-06-30, 16:04:36 UTC] {kubernetes_pod.py:433} INFO - Deleting pod:
airflow-test-pod-3528101c2e404b3dbf2cae3531523af7
[2022-06-30, 16:04:36 UTC] {taskinstance.py:1889} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py",
line 389, in execute
follow=True,
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/cncf/kubernetes/utils/pod_manager.py",
line 244, in fetch_container_logs
last_log_time = consume_logs(since_time=last_log_time, follow=follow)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/cncf/kubernetes/utils/pod_manager.py",
line 221, in consume_logs
follow=follow,
File
"/home/airflow/.local/lib/python3.7/site-packages/tenacity/__init__.py", line
324, in wrapped_f
return self(f, *args, **kw)
File
"/home/airflow/.local/lib/python3.7/site-packages/tenacity/__init__.py", line
404, in __call__
do = self.iter(retry_state=retry_state)
File
"/home/airflow/.local/lib/python3.7/site-packages/tenacity/__init__.py", line
360, in iter
raise retry_exc.reraise()
File
"/home/airflow/.local/lib/python3.7/site-packages/tenacity/__init__.py", line
193, in reraise
raise self.last_attempt.result()
File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 428, in
result
return self.__get_result()
File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 384, in
__get_result
raise self._exception
File
"/home/airflow/.local/lib/python3.7/site-packages/tenacity/__init__.py", line
407, in __call__
result = fn(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/cncf/kubernetes/utils/pod_manager.py",
line 331, in read_pod_logs
**additional_kwargs,
File
"/home/airflow/.local/lib/python3.7/site-packages/kubernetes/client/api/core_v1_api.py",
line 23747, in read_namespaced_pod_log
return self.read_namespaced_pod_log_with_http_info(name, namespace,
**kwargs) # noqa: E501
File
"/home/airflow/.local/lib/python3.7/site-packages/kubernetes/client/api/core_v1_api.py",
line 23880, in read_namespaced_pod_log_with_http_info
collection_formats=collection_formats)
File
"/home/airflow/.local/lib/python3.7/site-packages/kubernetes/client/api_client.py",
line 353, in call_api
_preload_content, _request_timeout, _host)
File
"/home/airflow/.local/lib/python3.7/site-packages/kubernetes/client/api_client.py",
line 184, in __call_api
_request_timeout=_request_timeout)
File
"/home/airflow/.local/lib/python3.7/site-packages/kubernetes/client/api_client.py",
line 377, in request
headers=headers)
File
"/home/airflow/.local/lib/python3.7/site-packages/kubernetes/client/rest.py",
line 244, in GET
query_params=query_params)
File
"/home/airflow/.local/lib/python3.7/site-packages/kubernetes/client/rest.py",
line 234, in request
raise ApiException(http_resp=r)
kubernetes.client.exceptions.ApiException: (422)
Reason: Unprocessable Entity
HTTP response headers: HTTPHeaderDict({'Audit-Id':
'f96bf433-3e0a-48d8-b4eb-a6559c4980b2', 'Cache-Control': 'no-cache, private',
'Content-Type': 'application/json', 'Date': 'Thu, 30 Jun 2022 16:04:34 GMT',
'Content-Length': '464'})
HTTP response body:
b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"PodLogOptions
\\"airflow-test-pod-3528101c2e404b3dbf2cae3531523af7\\" is invalid:
sinceSeconds: Invalid value: -105: must be greater than
0","reason":"Invalid","details":{"name":"airflow-test-pod-3528101c2e404b3dbf2cae3531523af7","kind":"PodLogOptions","causes":[{"reason":"FieldValueInvalid","message":"Invalid
value: -105: must be greater than 0","field":"sinceSeconds"}]},"code":422}\n'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py",
line 402, in execute
remote_pod=remote_pod,
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py",
line 425, in cleanup
f'Pod {pod and pod.metadata.name} returned a
failure:{error_message}\n{remote_pod}'
airflow.exceptions.AirflowException: Pod
airflow-test-pod-3528101c2e404b3dbf2cae3531523af7 returned a failure:
None
[2022-06-30, 16:04:36 UTC] {taskinstance.py:1400} INFO - Marking task as
FAILED. dag_id=Amadeus_MIDT, task_id=input_2_raw,
execution_date=20220630T155812, start_date=20220630T160428,
end_date=20220630T160436
[2022-06-30, 16:04:36 UTC] {standard_task_runner.py:97} ERROR - Failed to
execute job 28 for task input_2_raw (Pod
airflow-test-pod-3528101c2e404b3dbf2cae3531523af7 returned a failure:
None; 63)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
