alexkruc commented on code in PR #26886:
URL: https://github.com/apache/airflow/pull/26886#discussion_r992056119
##########
airflow/providers/amazon/aws/hooks/s3.py:
##########
@@ -902,14 +911,21 @@ def download_file(self, key: str, bucket_name: str | None
= None, local_path: st
else:
raise e
- with NamedTemporaryFile(dir=local_path, prefix='airflow_tmp_',
delete=False) as local_tmp_file:
+ if preserve_file_name:
+ local_dir = local_path if local_path else gettempdir()
Review Comment:
Eventually, I did 2 things:
- Added a check if the file exists before re-writing it, failing the task if
it already exists to bubble the issue to the user.
- Added another parameter, `use_autogenerated_subdir`, that is `True` by
default, which creates a new sub-directory. The user can disable it to control
the target file location, but it's on by default.
@o-nikolas @uranusjr @XD-DENG Will appreciate your review of the latest
additions to this flow 🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
