krisfur commented on code in PR #34627:
URL: https://github.com/apache/airflow/pull/34627#discussion_r1338839397
##########
airflow/providers/microsoft/azure/operators/container_instances.py:
##########
@@ -319,6 +319,9 @@ def _monitor_logging(self, resource_group: str, name: str)
-> int:
if state in ["Running", "Terminated", "Succeeded"]:
try:
logs = self._ci_hook.get_logs(resource_group, name)
+ if logs is None:
+ self.log.exception("Container log is broken,
marking as failed.")
Review Comment:
@Taragolis in issue #34516 at the end of the (admittedly long) error
traceback you will find that it doesn't fail the job, it just spits out the
errors (from getting a None log) and then puts the container back to waiting ->
pending -> running again.
I could not find anything in the code that would specifically cause such
restarts/reprovisioning to happen, so handling the error when container logs
are None is the only solution I could come up with.
I am not sure what running execute manually means so I can assume that's not
the case.
last bits of the traceback:
```
[2023-09-21, 13:27:10 UTC] {container_instances.py:344} ERROR - Exception
while getting container groups
Traceback (most recent call last):
File
"/home/azureuser/pyenv-airflow/lib/python3.8/site-packages/airflow/providers/microsoft/azure/operators/container_instances.py",
line 316, in _monitor_logging
logs = self._ci_hook.get_logs(resource_group, name)
File
"/home/azureuser/pyenv-airflow/lib/python3.8/site-packages/airflow/providers/microsoft/azure/hooks/container_instance.py",
line 119, in get_logs
return logs.content.splitlines(True)
AttributeError: 'NoneType' object has no attribute 'splitlines'
[2023-09-21, 13:27:11 UTC] {container_instances.py:311} INFO - Container
group state changed to Waiting
[2023-09-21, 13:27:13 UTC] {container_instances.py:311} INFO - Container
group state changed to Running
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
