Repository: libcloud Updated Branches: refs/heads/trunk f70264647 -> 4732d83be
Ensure RawResponse downloads don't consume RAM The fix is general but the cause is very specific: since libcloud 2.0, `download_object()` in the S3 and GCS drivers first places the item in RAM, then writes it to a file. The reason it broke is that accessing `body` in a requests Response consumes it entirely and loads it in RAM. To fix this, I moved `body` to a property: it is still accessibly, but loaded only if requested. Closes #1132 Signed-off-by: Tomaz Muraus <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/31411884 Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/31411884 Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/31411884 Branch: refs/heads/trunk Commit: 31411884cd2f989421e378f0bc8fc0c7b6c4783f Parents: f702646 Author: Quentin Pradet <[email protected]> Authored: Mon Oct 16 14:58:45 2017 +0400 Committer: Tomaz Muraus <[email protected]> Committed: Sun Jan 7 18:16:32 2018 +0100 ---------------------------------------------------------------------- libcloud/common/base.py | 6 +++++- libcloud/http.py | 4 ++++ 2 files changed, 9 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/libcloud/blob/31411884/libcloud/common/base.py ---------------------------------------------------------------------- diff --git a/libcloud/common/base.py b/libcloud/common/base.py index 24fdbdb..6dc0589 100644 --- a/libcloud/common/base.py +++ b/libcloud/common/base.py @@ -274,12 +274,15 @@ class RawResponse(Response): if not self._response: response = self.connection.connection.getresponse() self._response = HttpLibResponseProxy(response) - self.body = response.content if not self.success(): self.parse_error() return self._response @property + def body(self): + return self.response.body + + @property def reason(self): if not self._reason: self._reason = self.response.reason @@ -581,6 +584,7 @@ class Connection(object): url=url, body=data, headers=headers, + raw=raw, stream=stream) else: if retry_enabled: http://git-wip-us.apache.org/repos/asf/libcloud/blob/31411884/libcloud/http.py ---------------------------------------------------------------------- diff --git a/libcloud/http.py b/libcloud/http.py index 9149995..5e85026 100644 --- a/libcloud/http.py +++ b/libcloud/http.py @@ -319,3 +319,7 @@ class HttpLibResponseProxy(object): def version(self): # requests doesn't expose this return '11' + + @property + def body(self): + return self._response.content
