ashb commented on code in PR #48107:
URL: https://github.com/apache/airflow/pull/48107#discussion_r2021184888
##########
providers/google/src/airflow/providers/google/cloud/transfers/sftp_to_gcs.py:
##########
@@ -73,6 +73,11 @@ class SFTPToGCSOperator(BaseOperator):
Service Account Token Creator IAM role to the directly preceding
identity, with first
account from the list granting this role to the originating account
(templated).
:param sftp_prefetch: Whether to enable SFTP prefetch, the default is True.
+ :param use_stream: Determines the method of file transfer between SFTP and
GCS.
+ - If set to False (default), the file is downloaded to the worker's local
storage and
+ then uploaded to GCS. This may require significant disk space on the
worker for large files.
+ - If set to True, the file is streamed directly from SFTP to GCS, which
does not consume
+ local disk space on the worker.
Review Comment:
```suggestion
:param use_stream: Determines the method of file transfer between SFTP
and GCS.
- If set to False (default), the file is downloaded to the worker's
local storage and
then uploaded to GCS. This may require significant disk space on
the worker for large files.
- If set to True, the file is streamed directly from SFTP to GCS,
which does not consume
local disk space on the worker.
```
Sphinx can be picky about indentation
##########
providers/google/src/airflow/providers/google/cloud/hooks/gcs.py:
##########
@@ -726,6 +726,14 @@ def delete(self, bucket_name: str, object_name: str) ->
None:
self.log.info("Blob %s deleted.", object_name)
+ def get_bucket(self, bucket_name: str) -> storage.Bucket:
+ """
+ Get a bucket object from the Google Cloud Storage.
Review Comment:
```suggestion
Get a bucket object from the Google Cloud Storage.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
