Hi Noel, One thing to try could perhaps be the vcloud driver which predates vCloud 1.5 overhaul. This is due to the fact we modified a couple of shared chunks of code between non-1.5 and 1.5 versions of the vcloud driver. This could indicate whether it's a regression problem we were unable to test out or the Terremark service no longer works with the original driver version (assuming it worked before - which it should have).
Try revision 1210045 of vcloud.py with Python 2.7.x and see if it behaves the same way: https://svn.apache.org/viewvc/libcloud/trunk/libcloud/compute/drivers/vcloud.py?view=log That revision is just before we started modifications for vCloud 1.5. I hope this helps us progress further. On 26 June 2012 08:59, Noel Milton Vega <nmv...@computingarchitects.com> wrote: > Hi Sengor: > > Thanks for the replies. > > I manually patched vcloud.py by appending ".decide('utf-8')" in two > the places needing it, and that worked (... and makes sense ... I had > to the same decode() step in my non-libcloud httplib2/ReST case for > python-3). > > So let me move onto the subsequent parsing exceptions I mentioned... > which is raised during processing of the HTTP XML Response > (indicating malformed). Tracking it down, this happens in > vcloud.py, here: > > ================= > class VCloudConnection(ConnectionUserAndKey): > <snip> > > def _get_auth_token(self): > ... > <snip> > body = ET.XML(resp.read()) <--- > <snip> > ... > <snip> > ================= > > > Through various print() and type() statements, I see that "resp.read()", which > when expanded translates to "http.client.HTTPResponse.read()", returns a > bytes string. However, I'm wondering if this may also need to be similarly > ".decode('utf-8')", like this: > > body = ET.XML(resp.read().decode('utf-8')) > > > since the Python-3 documentation for "xml.etree.ElementTree.XML()" > (which is what ET.XML() expands to), indicates that it is expecting a string > (not a bytes encoded string). > > So I tried the modification above. Here is the before and after: > > resp.read() [ BEFORE .decode() inserted ] > > <class 'bytes'> <--- types is 'bytes', and the following is the actual bytes > string... > > b'b\'<OrgList xmlns="http://www.vmware.com/vcloud/v0.8"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xmlns:xsd="http://www.w3.org/2001/XMLSchema";>\\n <Org > href="https://services.vcloudexpress.terremark.com/api/v0.8/org/xxxx"; > type="application/vnd.vmware.vcloud.org+xml" > name="em...@example.com"/>\\n</OrgL' > > > resp.read().decode('utf-8') [ AFTER .decode() inserted ] > > > <class 'str'> <--- types is 'str', and the following is the actual string... > > b'<OrgList xmlns="http://www.vmware.com/vcloud/v0.8"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xmlns:xsd="http://www.w3.org/2001/XMLSchema";>\n <Org > href="https://services.vcloudexpress.terremark.com/api/v0.8/org/xxxx"; > type="application/vnd.vmware.vcloud.org+xml" > name="em...@example.com"/>\n</OrgL > > > Looking carefully at both the bytes string (un-decoded) and the string > (decoded) above -- forgetting about whether we > decode() it or not for the moment -- both sequences appear to be malformed... > They both have an extra "b'" > pre-pending it, and both are truncated at the end (i.e. the enclosing > "</OrgL" ... doesn't end correctly). Note: Just > > in case this was a print() artefact, I also wrote the output to a file (same > truncation issue). > > This above issue is only true for python-3. Python 2, (decode()ed or not) > emits output correctly as: > > <OrgList xmlns="http://www.vmware.com/vcloud/v0.8"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xmlns:xsd="http://www.w3.org/2001/XMLSchema";> <Org > href="https://services.vcloudexpress.terremark.com/api/v0.8/org/xxxx"; > type="application/vnd.vmware.vcloud.org+xml" > name="em...@example.com"/></OrgList> > > > So there appears to be two problems here: > (1) missing ".decode('utf-8')" method call. > (2) the output of "resp.read()", with or without the ".decode('utf-8')" > part, appears to be malformed. > > > To make matters more interesting, note that the python-3 case curl(1) output > shows a result that is not truncated, > but still, however, has the pre-pending "b'" (so it's still malformed). And > here is that curl(1) output > (remembering that the above were from print() debug statements of mine, not > from curl(1)): > > b'<OrgList xmlns="http://www.vmware.com/vcloud/v0.8"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xmlns:xsd="http://www.w3.org/2001/XMLSchema";>\n <Org > href="https://services.vcloudexpress.terremark.com/api/v0.8/org/xxxx"; > type="application/vnd.vmware.vcloud.org+xml" > name="em...@example.com"/>\n</OrgList>' > > > As I look more for the cause, any helpful feedback is appreciated. > > > Again,... the libcloud API is v0.10.1 with Python v3.2.3 > > > Thank you, > Noel > > >>________________________________ >> From: Sengor <seng...@gmail.com> >>To: users@libcloud.apache.org; Noel Milton Vega >><nmv...@computingarchitects.com> >>Sent: Thursday, June 21, 2012 10:21 PM >>Subject: Re: Terremark authentication example using libcloud ... >> >>Hi Noel, >> >>Replies in line below: >> >>On 22 June 2012 09:30, Noel Milton Vega <nmv...@computingarchitects.com> >>wrote: >>> (1) I was surprised that that base64 line worked at all (even in Python 2) >>> since I always thought use of "b" only worked for string literals >>> (not generated strings). >> >>Patch has been applied to trunk which now works for both Python 2 & 3. >>For more info see: https://issues.apache.org/jira/browse/LIBCLOUD-204 >> >> >>> (2) Not an issue for this specific thread, but even when I hard-coded >>> my auth string (just to get past the Python-3 related auth issue), sadly >>> another unrelated (xml parsing) issue arose almost immediately. >>> Just as a heads up, it was this: >>> >>> >>> Traceback (most recent call last): >>> File "/usr/lib64/python3.2/xml/etree/ElementTree.py", line 1668, in feed >>> self._parser.Parse(data, 0) >>> xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, >>> column 1 >>> >>> (So I never got my list_images() output.) >>> >>> (3) Similar to the last point, python-2 gets much further along, but also >>> eventually experiences an parse-related exception too (as it's >>> iterating through the list the of images too I believe). Again, as a >>> heads-up fyi: >>> >>> >>> File "/usr/lib/python2.7/site-packages/libcloud/common/base.py", line 76, >>> in __init__ >>> raise Exception(self.parse_error()) >>> Exception: <Element '{http://www.w3.org/1999/xhtml}html'; at 0x118dd10> >> >>Hard to say what the issue could be for these two. Could you post the >>debugging output for these two (don't forget to remove the private >>contents such as base64 encoded auth): >>http://libcloud.apache.org/docs/debugging.html >> >>Note I've only used the vCloud module under Python 2.x at the moment >>and don't have access to Terremark. We did slightly change the VDC >>object model when vCloud v1.5 support has been introduced. If this is >>related to what you see it'd be good to fix it up, otherwise it's >>something else getting in the way. >> >>-- >>sengork >> >> >> -- sengork
