alexkruc commented on code in PR #26886:
URL: https://github.com/apache/airflow/pull/26886#discussion_r997224258
##########
airflow/providers/amazon/aws/hooks/s3.py:
##########
@@ -902,14 +924,30 @@ def download_file(self, key: str, bucket_name: str | None
= None, local_path: st
else:
raise e
- with NamedTemporaryFile(dir=local_path, prefix='airflow_tmp_',
delete=False) as local_tmp_file:
+ if preserve_file_name:
+ local_dir = local_path if local_path else gettempdir()
+ subdir = f"airflow_tmp_dir_{uuid4().hex[0:8]}" if
use_autogenerated_subdir else ""
+ filename_in_s3 = s3_obj.key.rsplit('/', 1)[-1]
+ file_path = Path(local_dir, subdir, filename_in_s3)
+
+ if file_path.is_file():
+ self.log.error("file '%s' already exists. Failing the task and
not overwriting it", file_path)
+ raise FileExistsError
+
+ file_path.parent.mkdir(exist_ok=True, parents=True)
+
+ file = open(file_path, 'wb')
+ else:
+ file = NamedTemporaryFile(dir=local_path, prefix='airflow_tmp_',
delete=False) # type: ignore
Review Comment:
Oh I see that you've edited and suggested using `BinaryIO`, but sadly the
pre-commit hook is still failing on the same thing:
```shell
Run mypy for
providers.................................................................Failed
- hook id: run-mypy
- exit code: 1
airflow/providers/amazon/aws/hooks/s3.py:941: error: Name "file" already
defined on line 939 [no-redef]
file: BinaryIO = NamedTemporaryFile(dir=local_path, prefix...
^
Found 1 error in 1 file (checked 1 source file)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
