tiranux opened a new pull request, #52850: URL: https://github.com/apache/airflow/pull/52850
resolves: #49801 related: #48107 Support for streaming was added for SFTP_to_GCS Operator. As mentioned in #49801 it is not actually working as the object received in the SFTP hook a BlobWriter. ``` ERROR - Task failed with exception: source="task" • TypeError: expected str, bytes or os. PathLike object, not BlobWriter File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py", line 875 in run File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py", line 1162 in _execute_task File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 397 in wrapper File "/opt/airflow/providers/google/src/airflow/providers/google/cloud/transfers/sftp_to_gcs.py", line 159 in execute File "/opt/airflow/providers/google/src/airflow/providers/google/cloud/transfers/sftp_to_gcs.py", line 180 in _copy_single_object File "/opt/airflow/providers/sftp/src/airflow/providers/sftp/hooks/sftp.py", line 292 in retrieve_file File "/usr/local/lib/python3.10/site-packages/paramiko/sftp_client.py", line 839 in get ``` While the code can be changed in the SFTP_to_GCS Operator by calling the conn.getfo() directly there, it would break the abstraction. I think we can make the SFTP hook more robust to manage the BlobWriter for GCS and throw an exception for unknown types. --- -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
