jmcarp opened a new pull request #13156: URL: https://github.com/apache/airflow/pull/13156
The google-cloud-storage client sets a timeout of 60s for all requests and defaults to uploading and downloading files in a single request. When working with large files or slow connections, users might need to either increase the timeout or set a chunk size so that the sdk uses chunked uploads and downloads. This patch adds optional chunk_size and timeout arguments to the upload and download methods of the gcs hook and pins google-cloud-storage to a version that supports timeouts for all applicable methods. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
