ephraimbuddy commented on a change in pull request #16002:
URL: https://github.com/apache/airflow/pull/16002#discussion_r637619547
##########
File path: airflow/providers/microsoft/azure/log/wasb_task_handler.py
##########
@@ -180,15 +180,20 @@ def wasb_write(self, log: str, remote_log_location: str,
append: bool = True) ->
the new log is appended to any existing logs.
:type append: bool
"""
- if append and self.wasb_log_exists(remote_log_location):
- old_log = self.wasb_read(remote_log_location)
- log = '\n'.join([old_log, log]) if old_log else log
-
try:
- self.hook.load_string(
- log,
- self.wasb_container,
- remote_log_location,
- )
+ if append and self.wasb_log_exists(remote_log_location):
+ self.hook.append_string(
+ log,
+ self.wasb_container,
+ remote_log_location
+ )
+ else:
+ self.hook.load_string(
+ log,
+ self.wasb_container,
+ remote_log_location,
+ blob_type = 'AppendBlob',
+ overwrite=True
+ )
Review comment:
This looks like a breaking change. Take, for example, you have a
BlockBlob log already in the container and you want to append to it, won't it
be rejected since this is AppendBlob?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
