pankajastro commented on issue #38003: URL: https://github.com/apache/airflow/issues/38003#issuecomment-1991780598
Ok, so I did some investigation. The POST_TERMINATION_TIMEOUT is set to 120 seconds, which means that the pod's logs will be available for retrieval for up to 120 seconds after the pod termination. https://github.com/apache/airflow/blob/8fc984873aab3424df0d44351da136e5c65b81e2/airflow/providers/cncf/kubernetes/operators/pod.py#L235 But your task is hanging in the trigger for more than 120 after pod termination and because of this log is not available. To test this you can try to reduce the `poll_interval` to maybe 60 seconds (since your script probably will be finshed quickly) and your logs should be available. I'll see if it makes sense to parametize it. Found one more small bug while debugging and created fixes https://github.com/apache/airflow/pull/38075 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
