perry2of5 commented on code in PR #41142:
URL: https://github.com/apache/airflow/pull/41142#discussion_r1702172040
##########
airflow/providers/microsoft/azure/operators/container_instances.py:
##########
@@ -86,6 +86,12 @@ class AzureContainerInstancesOperator(BaseOperator):
:param container_timeout: max time allowed for the execution of
the container instance.
:param tags: azure tags as dict of str:str
+ :param xcom_all: Control if logs are pushed to XCOM similarly to how
DockerOperator does.
Review Comment:
I see what you mean. Reading the existing azure container instance operator
code, it looks like _monitor_logging repeatedly pulls the entire log into
memory and then logs just the new messages in _log_last.
https://github.com/apache/airflow/blob/main/airflow/providers/microsoft/azure/operators/container_instances.py#L352
https://github.com/apache/airflow/blob/main/airflow/providers/microsoft/azure/operators/container_instances.py#L387
Based on this, the operator is _already_ pulling the entire log into memory
repeatedly. My modification doesn't make things worse, but it doesn't make
things better either. I'd have to think about how we could optimize this.
It seems like my change isn't making anything worse. Faint praise....
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
