Germandrummer92 opened a new pull request #1592:
URL: https://github.com/apache/libcloud/pull/1592
## Retry Raw Http Requests
### Description
We are currently facing some storage driver exceptions when uploading many
small files. Stacktrace (parts removed):
```self._driver.upload_object_via_stream( File
"/usr/local/lib/python3.8/site-packages/libcloud/storage/drivers/s3.py", line
754, in upload_object_via_stream return self._put_object(container=container,
object_name=object_name, File
"/usr/local/lib/python3.8/site-packages/libcloud/storage/drivers/s3.py", line
900, in _put_object result_dict = self._upload_object( File
"/usr/local/lib/python3.8/site-packages/libcloud/storage/base.py", line 837, in
_upload_object response.parse_error() File
"/usr/local/lib/python3.8/site-packages/libcloud/storage/drivers/s3.py", line
147, in parse_error raise LibcloudError('Unknown error. Status code: %d' %
(self.status), libcloud.common.types.LibcloudError: <LibcloudError in <class
'libcloud.storage.drivers.s3.S3StorageDriver'> 'Unknown error. Status code:
429'>```
This change also enables retrying the raw requests. Together with #1588 (for
which we have a workaround), this would solve our issue completely and we could
keep using libcloud.
I only added a call to retrying the raw requests.
### Status
done, ready for review
### Checklist (tick everything that applies)
- [X] [Code
linting](http://libcloud.readthedocs.org/en/latest/development.html#code-style-guide)
(required, can be done after the PR checks)
- [X] Documentation
- [X] [Tests](http://libcloud.readthedocs.org/en/latest/testing.html)
- [ ]
[ICLA](http://libcloud.readthedocs.org/en/latest/development.html#contributing-bigger-changes)
(required for bigger changes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
