flolas commented on a change in pull request #16002:
URL: https://github.com/apache/airflow/pull/16002#discussion_r637623609
##########
File path: airflow/providers/microsoft/azure/log/wasb_task_handler.py
##########
@@ -180,15 +180,20 @@ def wasb_write(self, log: str, remote_log_location: str,
append: bool = True) ->
the new log is appended to any existing logs.
:type append: bool
"""
- if append and self.wasb_log_exists(remote_log_location):
- old_log = self.wasb_read(remote_log_location)
- log = '\n'.join([old_log, log]) if old_log else log
-
try:
- self.hook.load_string(
- log,
- self.wasb_container,
- remote_log_location,
- )
+ if append and self.wasb_log_exists(remote_log_location):
+ self.hook.append_string(
+ log,
+ self.wasb_container,
+ remote_log_location
+ )
+ else:
+ self.hook.load_string(
+ log,
+ self.wasb_container,
+ remote_log_location,
+ blob_type = 'AppendBlob',
+ overwrite=True
+ )
Review comment:
@ephraimbuddy Maybe handling that case in the exception and converting
an BlockBlob to AppendBlob? What do u think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
