Oops... Disregard the last part of my email below -- my comment
about the the curl(1) output.
The curl(1) output shows a properly formatted bytes string returned in the
response. But, however, properly formatted that curl(1) output may be,
the problem is still as I described below (and we are not dealing with
curl(1) output, but python call output). Thanks.
----- Original Message -----
> From: Noel Milton Vega <nmv...@computingarchitects.com>
> To: "users@libcloud.apache.org" <users@libcloud.apache.org>
> Cc:
> Sent: Monday, June 25, 2012 6:59 PM
> Subject: Re: Terremark authentication example using libcloud ...
>
> Hi Sengor:
>
> Thanks for the replies.
>
> I manually patched vcloud.py by appending ".decide('utf-8')"
> in two
> the places needing it, and that worked (... and makes sense ... I had
> to the same decode() step in my non-libcloud httplib2/ReST case for
> python-3).
>
> So let me move onto the subsequent parsing exceptions I mentioned...
> which is raised during processing of the HTTP XML Response
> (indicating malformed). Tracking it down, this happens in
> vcloud.py, here:
>
> =================
> class VCloudConnection(ConnectionUserAndKey):
> <snip>
>
> def _get_auth_token(self):
> ...
> <snip>
> body = ET.XML(resp.read()) <---
> <snip>
> ...
> <snip>
> =================
>
>
> Through various print() and type() statements, I see that
> "resp.read()", which
> when expanded translates to "http.client.HTTPResponse.read()", returns
> a
> bytes string. However, I'm wondering if this may also need to be similarly
> ".decode('utf-8')", like this:
>
> body = ET.XML(resp.read().decode('utf-8'))
>
>
> since the Python-3 documentation for "xml.etree.ElementTree.XML()"
> (which is what ET.XML() expands to), indicates that it is expecting a string
> (not a bytes encoded string).
>
> So I tried the modification above. Here is the before and after:
>
> resp.read() [ BEFORE .decode() inserted ]
>
> <class 'bytes'> <--- types is 'bytes', and the
> following is the actual bytes string...
>
> b'b\'<OrgList xmlns="http://www.vmware.com/vcloud/v0.8";
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xmlns:xsd="http://www.w3.org/2001/XMLSchema";>\\n <Org
> href="https://services.vcloudexpress.terremark.com/api/v0.8/org/xxxx";
> type="application/vnd.vmware.vcloud.org+xml"
> name="em...@example.com"/>\\n</OrgL'
>
>
> resp.read().decode('utf-8') [ AFTER .decode() inserted ]
>
>
> <class 'str'> <--- types is 'str', and the following
> is the actual string...
>
> b'<OrgList xmlns="http://www.vmware.com/vcloud/v0.8";
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xmlns:xsd="http://www.w3.org/2001/XMLSchema";>\n <Org
> href="https://services.vcloudexpress.terremark.com/api/v0.8/org/xxxx";
> type="application/vnd.vmware.vcloud.org+xml"
> name="em...@example.com"/>\n</OrgL
>
>
> Looking carefully at both the bytes string (un-decoded) and the string
> (decoded)
> above -- forgetting about whether we
> decode() it or not for the moment -- both sequences appear to be malformed...
> They both have an extra "b'"
> pre-pending it, and both are truncated at the end (i.e. the enclosing
> "</OrgL" ... doesn't end correctly). Note: Just
>
> in case this was a print() artefact, I also wrote the output to a file (same
> truncation issue).
>
> This above issue is only true for python-3. Python 2, (decode()ed or not)
> emits
> output correctly as:
>
> <OrgList xmlns="http://www.vmware.com/vcloud/v0.8";
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xmlns:xsd="http://www.w3.org/2001/XMLSchema";> <Org
> href="https://services.vcloudexpress.terremark.com/api/v0.8/org/xxxx";
> type="application/vnd.vmware.vcloud.org+xml"
> name="em...@example.com"/></OrgList>
>
>
> So there appears to be two problems here:
> (1) missing ".decode('utf-8')" method call.
> (2) the output of "resp.read()", with or without the
> ".decode('utf-8')" part, appears to be malformed.
>
>
> To make matters more interesting, note that the python-3 case curl(1) output
> shows a result that is not truncated,
> but still, however, has the pre-pending "b'" (so it's still
> malformed). And here is that curl(1) output
> (remembering that the above were from print() debug statements of mine, not
> from
> curl(1)):
>
> b'<OrgList xmlns="http://www.vmware.com/vcloud/v0.8";
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xmlns:xsd="http://www.w3.org/2001/XMLSchema";>\n <Org
> href="https://services.vcloudexpress.terremark.com/api/v0.8/org/xxxx";
> type="application/vnd.vmware.vcloud.org+xml"
> name="em...@example.com"/>\n</OrgList>'
>
>
> As I look more for the cause, any helpful feedback is appreciated.
>
>
> Again,... the libcloud API is v0.10.1 with Python v3.2.3
>
>
> Thank you,
> Noel
>
>
>> ________________________________
>> From: Sengor <seng...@gmail.com>
>> To: users@libcloud.apache.org; Noel Milton Vega
> <nmv...@computingarchitects.com>
>> Sent: Thursday, June 21, 2012 10:21 PM
>> Subject: Re: Terremark authentication example using libcloud ...
>>
>> Hi Noel,
>>
>> Replies in line below:
>>
>> On 22 June 2012 09:30, Noel Milton Vega
> <nmv...@computingarchitects.com> wrote:
>>> (1) I was surprised that that base64 line worked at all (even in Python
> 2)
>>> since I always thought use of "b" only worked for string
> literals
>>> (not generated strings).
>>
>> Patch has been applied to trunk which now works for both Python 2 & 3.
>> For more info see: https://issues.apache.org/jira/browse/LIBCLOUD-204
>>
>>
>>> (2) Not an issue for this specific thread, but even when I hard-coded
>>> my auth string (just to get past the Python-3 related auth issue),
> sadly
>>> another unrelated (xml parsing) issue arose almost immediately.
>>> Just as a heads up, it was this:
>>>
>>>
>>> Traceback (most recent call last):
>>> File "/usr/lib64/python3.2/xml/etree/ElementTree.py", line
> 1668, in feed
>>> self._parser.Parse(data, 0)
>>> xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1,
> column 1
>>>
>>> (So I never got my list_images() output.)
>>>
>>> (3) Similar to the last point, python-2 gets much further along, but
> also
>>> eventually experiences an parse-related exception too (as it's
>>> iterating through the list the of images too I believe). Again, as
> a heads-up fyi:
>>>
>>>
>>> File
> "/usr/lib/python2.7/site-packages/libcloud/common/base.py", line 76,
> in __init__
>>> raise Exception(self.parse_error())
>>> Exception: <Element '{http://www.w3.org/1999/xhtml}html'; at
> 0x118dd10>
>>
>> Hard to say what the issue could be for these two. Could you post the
>> debugging output for these two (don't forget to remove the private
>> contents such as base64 encoded auth):
>> http://libcloud.apache.org/docs/debugging.html
>>
>> Note I've only used the vCloud module under Python 2.x at the moment
>> and don't have access to Terremark. We did slightly change the VDC
>> object model when vCloud v1.5 support has been introduced. If this is
>> related to what you see it'd be good to fix it up, otherwise it's
>> something else getting in the way.
>>
>> --
>> sengork
>>
>>
>>
>
