Repository: libcloud Updated Branches: refs/heads/trunk 77afd30dc -> e6a6676a5
PY3: Reduce AWS S3 test errors if lxml is present At least lxml should raise ValueError pretty quickly if self.body is a str type, and it was almost certainly assumed to be 'utf-8', this being the web, so we'll pay the price of the decode & encode on the production happy path to get a probable overall speedup from lxml. http://lxml.de/performance.html Seems cleaner than alternate approaches like trying to inject an additional Response.content attribute to work with the Python 3 bytes type or changing the response.body to a bytes type (causes JSON tests to break). e.g. See https://github.com/apache/libcloud/pull/767 I don't have an aliyun.py, etc driver to test against so don't feel comfortable changing that though happy for others to do so. In my local Python 3.5+lxml virtualenv, running "python setup.py test": BEFORE ---------------------------------------------------------------------- Ran 5439 tests in 24.415s FAILED (failures=4, errors=595, skipped=14) AFTER ---------------------------------------------------------------------- Ran 5439 tests in 27.036s FAILED (failures=4, errors=176, skipped=14) Closes #769 Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/e6a6676a Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/e6a6676a Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/e6a6676a Branch: refs/heads/trunk Commit: e6a6676a5753569541b914cae614819113a8b042 Parents: 77afd30 Author: Peter Schmidt <pe...@peterjs.com> Authored: Wed Apr 20 16:33:37 2016 +1000 Committer: anthony-shaw <anthonys...@apache.org> Committed: Sat Apr 23 13:11:09 2016 +1000 ---------------------------------------------------------------------- libcloud/common/base.py | 6 +++++- libcloud/test/storage/test_s3.py | 6 +++++- 2 files changed, 10 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/libcloud/blob/e6a6676a/libcloud/common/base.py ---------------------------------------------------------------------- diff --git a/libcloud/common/base.py b/libcloud/common/base.py index bda758b..318d28e 100644 --- a/libcloud/common/base.py +++ b/libcloud/common/base.py @@ -270,7 +270,11 @@ class XmlResponse(Response): return self.body try: - body = ET.XML(self.body) + try: + body = ET.XML(self.body) + except ValueError: + # lxml wants a bytes and tests are basically hard-coded to str + body = ET.XML(self.body.encode('utf-8')) except: raise MalformedResponseError('Failed to parse XML', body=self.body, http://git-wip-us.apache.org/repos/asf/libcloud/blob/e6a6676a/libcloud/test/storage/test_s3.py ---------------------------------------------------------------------- diff --git a/libcloud/test/storage/test_s3.py b/libcloud/test/storage/test_s3.py index 2a6e873..af44f6d 100644 --- a/libcloud/test/storage/test_s3.py +++ b/libcloud/test/storage/test_s3.py @@ -338,7 +338,11 @@ class S3MockRawResponse(MockRawResponse): return self.body try: - body = ET.XML(self.body) + try: + body = ET.XML(self.body) + except ValueError: + # lxml wants a bytes and tests are basically hard-coded to str + body = ET.XML(self.body.encode('utf-8')) except: raise MalformedResponseError("Failed to parse XML", body=self.body,
