moiseenkov commented on code in PR #28336:
URL: https://github.com/apache/airflow/pull/28336#discussion_r1059315589
##########
airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py:
##########
@@ -168,6 +168,7 @@ class KubernetesPodOperator(BaseOperator):
:param labels: labels to apply to the Pod. (templated)
:param startup_timeout_seconds: timeout in seconds to startup the pod.
:param get_logs: get the stdout of the container as logs of the tasks.
+ :param logs_timeout: timeout in seconds to read logs after container
termination.
Review Comment:
Well, the problem occurs when we read logs after a container is being
terminated. We researched this issue but the actual root cause wasn't found
because it seems that the problem is at a very low level (third-party library
`urllib3` hangs at this line:
https://github.com/urllib3/urllib3/blob/d393b4a5091c27d2e158074f81feb264c5c175af/src/urllib3/response.py#L999).
Experiments show that reading logs works fine for about 2-3 minutes after
container termination, but if not all logs were red then it hangs. Because we
didn't figure out what exactly causes hanging, we decided to make this
parameter configurable for cases if `urllib3`'s behavior will change in the
future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
