AshleyT3 opened a new issue, #1698: URL: https://github.com/apache/libcloud/issues/1698
## Summary The download_object_as_stream for Azure located in azure_blobs.py will silently ignore the caller's specified chunk_size. It should probably raise an exception if the caller's value is not acceptable. Generally, much of chunk-based upload/download seems dependent on knowing all chunks will equal chunk_size until the last chunk. If libcloud silently changes the caller's expectation, it can lead to confusion (see example in details below). ## Detailed Information [Below is snippet from azure_blobs.py](https://github.com/apache/libcloud/blob/f3aa980ee55ac914b340c5d129435357c30082c8/libcloud/storage/drivers/azure_blobs.py#L717) ... <pre> AZURE_DOWNLOAD_CHUNK_SIZE = ( int(os.getenv("LIBCLOUD_AZURE_DOWNLOAD_CHUNK_SIZE_MB", "4")) * 1024 * 1024 # 4194304 bytes ) ... def download_object_as_stream(self, obj, chunk_size=None): """ @inherits: :class:`StorageDriver.download_object_as_stream` """ obj_path = self._get_object_path(obj.container, obj.name) response = self.connection.request( obj_path, method="GET", stream=True, raw=True ) iterator = response.iter_content(<b>AZURE_DOWNLOAD_CHUNK_SIZE</b>) ... drv.download_object_as_stream(storage_object, <b>chunk_size=5*1024*1024</b>) # as one example </pre> In the above example, the caller specified 5MB but immediately receives a smaller chunk given silent chunk_size downgrade by libcloud. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
