[GitHub] libcloud pull request: [LIBCLOUD-707] Fixed exception when parsing...

2015-05-13 Thread chrisob
GitHub user chrisob opened a pull request:

https://github.com/apache/libcloud/pull/514

[LIBCLOUD-707] Fixed exception when parsing CloudSigma static IPs

Fixes issue with CloudSigma compute driver: 
https://issues.apache.org/jira/browse/LIBCLOUD-707

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chrisob/libcloud 
LIBCLOUD-707_cloudsigma_response_parsing_fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/514.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #514


commit 2a48a10f262f18c9e107b050dee464a9bae73886
Author: Chris O'Brien chriso...@gmail.com
Date:   2015-05-13T11:14:07Z

[LIBCLOUD-707] Fixed exception when parsing CloudSigma static IPs




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (LIBCLOUD-707) CloudSigma driver throws exception when parsing response in list_nodes

2015-05-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541741#comment-14541741
 ] 

ASF GitHub Bot commented on LIBCLOUD-707:
-

GitHub user chrisob opened a pull request:

https://github.com/apache/libcloud/pull/514

[LIBCLOUD-707] Fixed exception when parsing CloudSigma static IPs

Fixes issue with CloudSigma compute driver: 
https://issues.apache.org/jira/browse/LIBCLOUD-707

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chrisob/libcloud 
LIBCLOUD-707_cloudsigma_response_parsing_fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/514.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #514


commit 2a48a10f262f18c9e107b050dee464a9bae73886
Author: Chris O'Brien chriso...@gmail.com
Date:   2015-05-13T11:14:07Z

[LIBCLOUD-707] Fixed exception when parsing CloudSigma static IPs




 CloudSigma driver throws exception when parsing response in list_nodes
 --

 Key: LIBCLOUD-707
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-707
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Affects Versions: 0.14.0-beta3, 0.13.3, 0.15.0, 0.15.1
Reporter: Chris O'Brien

 While attempting to parse the IPs from a node's NICs in CloudSigma's API 
 response, libcloud throws an exception when the node's IP is static: 
 {noformat}
 TypeError: unhashable type: 'dict'
 File /home/dbs_support/dev/libcloudsigma/test.py, line 55, in module
   instances = deployer.getInstances()
 File /home/dbs_support/dev/libcloudsigma/test.py, line 44, in getInstances
   return self.driver.list_nodes()
 File /home/dbs_support/lib/libcloud/compute/drivers/cloudsigma.py, line 
 1026, in list_nodes
   nodes = [self._to_node(data=item) for item in response['objects']]
 File /home/dbs_support/lib/libcloud/compute/drivers/cloudsigma.py, line 
 1910, in _to_node
   _public_ips, _private_ips = self._parse_ips_from_nic(nic=nic)
 File /home/dbs_support/lib/libcloud/compute/drivers/cloudsigma.py, line 
 1888, in _parse_ips_from_nic
   ips = set(ips)
 {noformat}
 It seems that libcloud doesn't account for CloudSigma's response for static 
 IPs containing the 'uuid' as a subobject of 'ip' in the 'ip_v4_conf' for a 
 NIC.  
 For example, a NIC with a static IP belonging to a server:
 {noformat}
 {
 firewall_policy:null,
 boot_order:null,
 vlan:null,
 ip_v4_conf:{
 ip:{
 uuid:178.22.68.44,
 resource_uri:/api/2.0/ips/178.22.68.44/
 },
 conf:static
 },
 mac:22:9d:42:9d:30:2f,
 model:virtio,
 runtime:{
 interface_type:public,
 ip_v4:{
 uuid:178.22.68.44,
 resource_uri:/api/2.0/ips/178.22.68.44/
 },
 io:{
 bytes_sent:403849752,
 packets_sent:6089897,
 bytes_recv:1348149166,
 packets_recv:115
 },
 ip_v6:null
 },
 ip_v6_conf:null
 }
 {noformat}
 The NIC's {{\['ip_v4_conf'\]\['ip'\]}} object (which is a dict) gets put into 
 a list (ips) with other strings (representing IPs).  It chokes when 
 attempting to convert this list containing both dicts and strings into a set. 
  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


libcloud git commit: [google|compute] Add support for GCE list paging and filtering

2015-05-13 Thread erjohnso
Repository: libcloud
Updated Branches:
  refs/heads/trunk 582633929 - 1ed502045


[google|compute] Add support for GCE list paging and filtering

GCE will return a maximum of 500 resources in a single list. This change
adds an iterator that allows filtering and/or paging of list results.

Closes #491

Signed-off-by: Eric Johnson erjoh...@google.com


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/1ed50204
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/1ed50204
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/1ed50204

Branch: refs/heads/trunk
Commit: 1ed5020450e6e8deb0361239bfb91ba795447e87
Parents: 5826339
Author: Lee Verberne v...@google.com
Authored: Tue Jan 6 09:54:05 2015 -0800
Committer: Eric Johnson erjoh...@google.com
Committed: Wed May 13 13:52:46 2015 +

--
 CHANGES.rst |   4 +
 libcloud/compute/drivers/gce.py | 174 ++-
 .../compute/fixtures/gce/regions-paged-1.json   |  97 +++
 .../compute/fixtures/gce/regions-paged-2.json   |  52 ++
 libcloud/test/compute/fixtures/gce/zones.json   |  12 +-
 libcloud/test/compute/test_gce.py   |  63 ++-
 6 files changed, 393 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/1ed50204/CHANGES.rst
--
diff --git a/CHANGES.rst b/CHANGES.rst
index 98ba7a4..5b7ef6f 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -25,6 +25,10 @@ General
 Compute
 ~~~
 
+- Google Compute now supports paginated lists including filtering.
+  (GITHUB-491)
+  [Lee Verberne]
+
 - OpenStackNodeSize objects now support optional, additional fields that are
   supported in OpenStack 2.1: `ephemeral_disk`, `swap`, `extra`.
   (GITHUB-488, LIBCLOUD-682)

http://git-wip-us.apache.org/repos/asf/libcloud/blob/1ed50204/libcloud/compute/drivers/gce.py
--
diff --git a/libcloud/compute/drivers/gce.py b/libcloud/compute/drivers/gce.py
index 666f1c1..e1e7549 100644
--- a/libcloud/compute/drivers/gce.py
+++ b/libcloud/compute/drivers/gce.py
@@ -64,7 +64,28 @@ class GCEResponse(GoogleResponse):
 
 
 class GCEConnection(GoogleBaseConnection):
-Connection class for the GCE driver.
+
+Connection class for the GCE driver.
+
+GCEConnection extends :class:`google.GoogleBaseConnection` for 2 reasons:
+  1. modify request_path for GCE URI.
+  2. Implement gce_params functionality described below.
+
+If the parameter gce_params is set to a dict prior to calling request(),
+the URL parameters will be updated to include those key/values FOR A
+SINGLE REQUEST. If the response contains a nextPageToken,
+gce_params['pageToken'] will be set to its value. This can be used to
+implement paging in list:
+
+ params, more_results = {'maxResults': 2}, True
+ while more_results:
+... driver.connection.gce_params=params
+... driver.ex_list_urlmaps()
+... more_results = 'pageToken' in params
+...
+[GCEUrlMap id=... name=cli-map, GCEUrlMap id=... name=lc-map]
+[GCEUrlMap id=... name=web-map]
+
 host = 'www.googleapis.com'
 responseCls = GCEResponse
 
@@ -76,6 +97,138 @@ class GCEConnection(GoogleBaseConnection):
 **kwargs)
 self.request_path = '/compute/%s/projects/%s' % (API_VERSION,
  project)
+self.gce_params = None
+
+def pre_connect_hook(self, params, headers):
+
+Update URL parameters with values from self.gce_params.
+
+@inherits: :class:`GoogleBaseConnection.pre_connect_hook`
+
+params, headers = super(GCEConnection, self).pre_connect_hook(params,
+  headers)
+if self.gce_params:
+params.update(self.gce_params)
+return params, headers
+
+def request(self, *args, **kwargs):
+
+Perform request then do GCE-specific processing of URL params.
+
+@inherits: :class:`GoogleBaseConnection.request`
+
+response = super(GCEConnection, self).request(*args, **kwargs)
+
+# If gce_params has been set, then update the pageToken with the
+# nextPageToken so it can be used in the next request.
+if self.gce_params:
+if 'nextPageToken' in response.object:
+self.gce_params['pageToken'] = response.object['nextPageToken']
+elif 'pageToken' in self.gce_params:
+del self.gce_params['pageToken']
+self.gce_params = None
+
+return response
+
+
+class 

[GitHub] libcloud pull request: Add support for GCE list paging and filteri...

2015-05-13 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/491


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Resolved] (LIBCLOUD-657) Update Google auth to support new JSON key format

2015-05-13 Thread Eric Johnson (JIRA)

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Johnson resolved LIBCLOUD-657.
---
Resolution: Implemented

 Update Google auth to support new JSON key format
 -

 Key: LIBCLOUD-657
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-657
 Project: Libcloud
  Issue Type: New Feature
  Components: Compute
Reporter: Eric Johnson
Assignee: Eric Johnson

 Google introduced a new Service Account key format (JSON) and over time this 
 will likely become the preferred format. We should begin by supporting this 
 new format soon. If Google does make this the default, we may want to 
 follow-up and issue a (debug | info) warning to users relying on P12 format.
 AFAIK, there is no clear documentation on this format. However, if you open 
 up the Developers Console and look under APIs  Auth - Credentials, 
 you'll see the Generate New JSON Key button next to any existing Service 
 Accounts (or be given the option to create one with a new SA).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (LIBCLOUD-627) service account auth fails with gce driver on python3

2015-05-13 Thread Eric Johnson (JIRA)

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Johnson resolved LIBCLOUD-627.
---
Resolution: Implemented

 service account auth fails with gce driver on python3
 -

 Key: LIBCLOUD-627
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-627
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Siim Põder
Assignee: Eric Johnson
Priority: Minor
  Labels: Google

 Fails due to string/binary confusion in GoogleServiceAcctAuthConnection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (LIBCLOUD-709) Implement retry on rate limit for nova api

2015-05-13 Thread Arpith Prakash (JIRA)
Arpith Prakash created LIBCLOUD-709:
---

 Summary: Implement retry on rate limit for nova api
 Key: LIBCLOUD-709
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-709
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Arpith Prakash
Priority: Trivial


Nova client raises exception when there are too many request, and retry-after 
time is being sent within the header. We should be catching that exception and 
retry automatically which could resolve most of downstream additional logic of 
implement it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (LIBCLOUD-699) GCE Compute driver cannot accept ex_nic_gce_struct for create_node

2015-05-13 Thread Eric Johnson (JIRA)

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Johnson reassigned LIBCLOUD-699:
-

Assignee: Eric Johnson

 GCE Compute driver cannot accept ex_nic_gce_struct for create_node
 --

 Key: LIBCLOUD-699
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-699
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 If you specify the {{ex_nic_gce_struct}} param for the {{create_node}} or 
 {{ex_create_multiple_nodes}}
 Any value other than {{None}} and the test for {{if network and 
 ex_nic_gce_struct:}} in {{_create_node_req}} fails due to network having a 
 default value of 'default'.
 As a workaround I tried to set network to '', since that would have caused it 
 to pass the comparison test, however that leads to an error from the 
 following snippet:
 {code}
 if not hasattr(ex_network, 'name'):
 ex_network = self.ex_get_network(ex_network)
 {code}
 Which returns with the following error:
 {code}
 error: {'locationType': 'parameter', 'domain': 'global', 'message': Invalid 
 value 'None'. Values must match the following regular expression: 
 '[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?', 'reason': 'invalidParameter', 
 'location': 'network'}
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (LIBCLOUD-697) GCE Compute driver create_node does not accept str for ex_boot_disk

2015-05-13 Thread Eric Johnson (JIRA)

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Johnson reassigned LIBCLOUD-697:
-

Assignee: Eric Johnson

 GCE Compute driver create_node does not accept str for ex_boot_disk
 ---

 Key: LIBCLOUD-697
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-697
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 The documentation for the GCE compute driver states that the ex_boot_disk 
 param for create_node takes either a StorageVolume or str.  If you pass a 
 string, it results in the following traceback:
 {code:none}
 failed: [localhost] = {failed: true, parsed: false}
 Traceback (most recent call last):
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 2278, in module
 main()
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 577, in main
 module.exit_json(**(node_mgr.execute()))
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 352, in execute
 metadata)
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 461, in create_instances_by_names
 lc_network, lc_zone, metadata)
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 433, in create_instance_by_name
 ex_nic_gce_struct=nics
   File /usr/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 2202, in create_node
 ex_automatic_restart)
   File /usr/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 4507, in _create_node_req
 'deviceName': boot_disk.name,
 AttributeError: 'str' object has no attribute 'name'
 {code}
 The issue appears to be that create_node is passing the ex_boot_disk argument 
 directly to _create_node_req (where the boot_disk param expects a 
 StorageVolume or None).
 Either the docs need to be updated or create_node should instantiate a 
 StorageVolume from ex_boot_disk if it is a str.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (LIBCLOUD-697) GCE Compute driver create_node does not accept str for ex_boot_disk

2015-05-13 Thread Eric Johnson (JIRA)

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Johnson resolved LIBCLOUD-697.
---
Resolution: Implemented

 GCE Compute driver create_node does not accept str for ex_boot_disk
 ---

 Key: LIBCLOUD-697
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-697
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 The documentation for the GCE compute driver states that the ex_boot_disk 
 param for create_node takes either a StorageVolume or str.  If you pass a 
 string, it results in the following traceback:
 {code:none}
 failed: [localhost] = {failed: true, parsed: false}
 Traceback (most recent call last):
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 2278, in module
 main()
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 577, in main
 module.exit_json(**(node_mgr.execute()))
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 352, in execute
 metadata)
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 461, in create_instances_by_names
 lc_network, lc_zone, metadata)
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 433, in create_instance_by_name
 ex_nic_gce_struct=nics
   File /usr/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 2202, in create_node
 ex_automatic_restart)
   File /usr/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 4507, in _create_node_req
 'deviceName': boot_disk.name,
 AttributeError: 'str' object has no attribute 'name'
 {code}
 The issue appears to be that create_node is passing the ex_boot_disk argument 
 directly to _create_node_req (where the boot_disk param expects a 
 StorageVolume or None).
 Either the docs need to be updated or create_node should instantiate a 
 StorageVolume from ex_boot_disk if it is a str.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (LIBCLOUD-687) Unable to set GCE metadata if no metadata already exists

2015-05-13 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542681#comment-14542681
 ] 

ASF subversion and git services commented on LIBCLOUD-687:
--

Commit aaa8e9e3a3a73d44aa6632089d60e8806f0f2de4 in libcloud's branch 
refs/heads/trunk from [~erjohnso]
[ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=aaa8e9e ]

[google compute] Fixes LIBCLOUD-687: handle empty project metadata

Closes #521

Signed-off-by: Eric Johnson erjoh...@google.com


 Unable to set GCE metadata if no metadata already exists
 

 Key: LIBCLOUD-687
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-687
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Eli Courtwright
Assignee: Eric Johnson

 The {{libcloud.compute.drivers.gce.GCENodeDriver}} class has a nonstandard 
 method {{ex_set_common_metadata}} which we can use to (among other things) 
 set SSH keys.
 When calling this method on the latest libcloud 0.17.0 (which is not 
 currently a listed version in Jira), I get the following error:
 {code}
  conn.ex_set_common_instance_metadata({})
 Traceback (most recent call last):
   File stdin, line 1, in module
   File 
 /opt/sideboard/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 1129, in ex_set_common_instance_metadata
 for md in current_metadata['items']:
 KeyError: 'items'
 {code}
 I get this error regardless of what data I pass to the 
 {{ex_set_common_instance_metadata}} method.  From delving into the libcloud 
 code, I find the following block of code:
 {code}
 project = self.ex_get_project()
 current_metadata = project.extra['commonInstanceMetadata']
 fingerprint = current_metadata['fingerprint']
 # grab copy of current 'sshKeys' in case we want to retain them
 current_keys = 
 for md in current_metadata['items']:
 {code}
 So it looks as if the method assumes that {{self.ex_get_project()}} will 
 return:
 - an object with a {{.extra}} dictionary (which it does)
 - that dictionary will have a commonInstanceMetadata key which maps to 
 another dictionary (which is correct)
 - that sub-dictionary will have an items key whose value is iterable (which 
 is wrong)
 That last assumption is only wrong if there is no project metadata already 
 associated with the project, e.g. for a new project.
 I'm working around this problem currently by monkeypatching libcloud:
 {code}
 def patch_gce_ex_get_project():
 orig_get_project = GCENodeDriver.ex_get_project
 def ex_get_project(self):
 project = orig_get_project(self)
 project.extra['commonInstanceMetadata'].setdefault('items', [])
 return project
 GCENodeDriver.ex_get_project = ex_get_project
 {code}
 but having a real fix would be great for the next version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


libcloud git commit: [google compute] Fixes LIBCLOUD-687: handle empty project metadata

2015-05-13 Thread erjohnso
Repository: libcloud
Updated Branches:
  refs/heads/trunk 8664118c8 - aaa8e9e3a


[google compute] Fixes LIBCLOUD-687: handle empty project metadata

Closes #521

Signed-off-by: Eric Johnson erjoh...@google.com


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/aaa8e9e3
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/aaa8e9e3
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/aaa8e9e3

Branch: refs/heads/trunk
Commit: aaa8e9e3a3a73d44aa6632089d60e8806f0f2de4
Parents: 8664118
Author: Eric Johnson erjoh...@google.com
Authored: Wed May 13 20:31:45 2015 +
Committer: Eric Johnson erjoh...@google.com
Committed: Wed May 13 20:45:23 2015 +

--
 libcloud/compute/drivers/gce.py | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/aaa8e9e3/libcloud/compute/drivers/gce.py
--
diff --git a/libcloud/compute/drivers/gce.py b/libcloud/compute/drivers/gce.py
index 4f15715..90c1e05 100644
--- a/libcloud/compute/drivers/gce.py
+++ b/libcloud/compute/drivers/gce.py
@@ -1300,10 +1300,13 @@ class GCENodeDriver(NodeDriver):
 project = self.ex_get_project()
 current_metadata = project.extra['commonInstanceMetadata']
 fingerprint = current_metadata['fingerprint']
+md_items = []
+if 'items' in current_metadata:
+md_items = current_metadata['items']
 
 # grab copy of current 'sshKeys' in case we want to retain them
 current_keys = 
-for md in current_metadata['items']:
+for md in md_items:
 if md['key'] == 'sshKeys':
 current_keys = md['value']
 



[GitHub] libcloud pull request: [google compute] handle empty project metad...

2015-05-13 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/521


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (LIBCLOUD-699) GCE Compute driver cannot accept ex_nic_gce_struct for create_node

2015-05-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542523#comment-14542523
 ] 

ASF GitHub Bot commented on LIBCLOUD-699:
-

Github user erjohnso closed the pull request at:

https://github.com/apache/libcloud/pull/518


 GCE Compute driver cannot accept ex_nic_gce_struct for create_node
 --

 Key: LIBCLOUD-699
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-699
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 If you specify the {{ex_nic_gce_struct}} param for the {{create_node}} or 
 {{ex_create_multiple_nodes}}
 Any value other than {{None}} and the test for {{if network and 
 ex_nic_gce_struct:}} in {{_create_node_req}} fails due to network having a 
 default value of 'default'.
 As a workaround I tried to set network to '', since that would have caused it 
 to pass the comparison test, however that leads to an error from the 
 following snippet:
 {code}
 if not hasattr(ex_network, 'name'):
 ex_network = self.ex_get_network(ex_network)
 {code}
 Which returns with the following error:
 {code}
 error: {'locationType': 'parameter', 'domain': 'global', 'message': Invalid 
 value 'None'. Values must match the following regular expression: 
 '[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?', 'reason': 'invalidParameter', 
 'location': 'network'}
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (LIBCLOUD-687) Unable to set GCE metadata if no metadata already exists

2015-05-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542682#comment-14542682
 ] 

ASF GitHub Bot commented on LIBCLOUD-687:
-

Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/521


 Unable to set GCE metadata if no metadata already exists
 

 Key: LIBCLOUD-687
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-687
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Eli Courtwright
Assignee: Eric Johnson

 The {{libcloud.compute.drivers.gce.GCENodeDriver}} class has a nonstandard 
 method {{ex_set_common_metadata}} which we can use to (among other things) 
 set SSH keys.
 When calling this method on the latest libcloud 0.17.0 (which is not 
 currently a listed version in Jira), I get the following error:
 {code}
  conn.ex_set_common_instance_metadata({})
 Traceback (most recent call last):
   File stdin, line 1, in module
   File 
 /opt/sideboard/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 1129, in ex_set_common_instance_metadata
 for md in current_metadata['items']:
 KeyError: 'items'
 {code}
 I get this error regardless of what data I pass to the 
 {{ex_set_common_instance_metadata}} method.  From delving into the libcloud 
 code, I find the following block of code:
 {code}
 project = self.ex_get_project()
 current_metadata = project.extra['commonInstanceMetadata']
 fingerprint = current_metadata['fingerprint']
 # grab copy of current 'sshKeys' in case we want to retain them
 current_keys = 
 for md in current_metadata['items']:
 {code}
 So it looks as if the method assumes that {{self.ex_get_project()}} will 
 return:
 - an object with a {{.extra}} dictionary (which it does)
 - that dictionary will have a commonInstanceMetadata key which maps to 
 another dictionary (which is correct)
 - that sub-dictionary will have an items key whose value is iterable (which 
 is wrong)
 That last assumption is only wrong if there is no project metadata already 
 associated with the project, e.g. for a new project.
 I'm working around this problem currently by monkeypatching libcloud:
 {code}
 def patch_gce_ex_get_project():
 orig_get_project = GCENodeDriver.ex_get_project
 def ex_get_project(self):
 project = orig_get_project(self)
 project.extra['commonInstanceMetadata'].setdefault('items', [])
 return project
 GCENodeDriver.ex_get_project = ex_get_project
 {code}
 but having a real fix would be great for the next version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (LIBCLOUD-621) Google Compute Engine Driver: ex_set_volume_auto_delete method don't work with volume parameter

2015-05-13 Thread Eric Johnson (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542608#comment-14542608
 ] 

Eric Johnson commented on LIBCLOUD-621:
---

This appears to be fixed. Please re-open if not the case.

 Google Compute Engine Driver: ex_set_volume_auto_delete method don't work 
 with volume parameter
 ---

 Key: LIBCLOUD-621
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-621
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Matthias Wiesner
Assignee: Eric Johnson
  Labels: Google

 On passing a StorageVolume as parameter one get the error:
 libcloud.common.google.InvalidRequestError: {u'locationType': u'parameter', 
 u'domain': u'global', u'message': uInvalid value 'StorageVolume 
 id=17316053985365710060 size=200 driver=Google Compute Engine'. Values must 
 match the following regular expression: '\\w[\\w.-]{0,254}', u'reason': 
 u'invalidParameter', u'location': u'deviceName'}
 Solution:
 rewrite:
 {{'deviceName': volume,}}
 to:
 {{'deviceName': volume.name,}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (LIBCLOUD-621) Google Compute Engine Driver: ex_set_volume_auto_delete method don't work with volume parameter

2015-05-13 Thread Eric Johnson (JIRA)

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Johnson resolved LIBCLOUD-621.
---
Resolution: Fixed

 Google Compute Engine Driver: ex_set_volume_auto_delete method don't work 
 with volume parameter
 ---

 Key: LIBCLOUD-621
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-621
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Matthias Wiesner
Assignee: Eric Johnson
  Labels: Google

 On passing a StorageVolume as parameter one get the error:
 libcloud.common.google.InvalidRequestError: {u'locationType': u'parameter', 
 u'domain': u'global', u'message': uInvalid value 'StorageVolume 
 id=17316053985365710060 size=200 driver=Google Compute Engine'. Values must 
 match the following regular expression: '\\w[\\w.-]{0,254}', u'reason': 
 u'invalidParameter', u'location': u'deviceName'}
 Solution:
 rewrite:
 {{'deviceName': volume,}}
 to:
 {{'deviceName': volume.name,}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (LIBCLOUD-697) GCE Compute driver create_node does not accept str for ex_boot_disk

2015-05-13 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542590#comment-14542590
 ] 

ASF subversion and git services commented on LIBCLOUD-697:
--

Commit 8664118c877a43f1434a576bd46223707589b078 in libcloud's branch 
refs/heads/trunk from [~erjohnso]
[ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=8664118 ]

[google compute] fixes LIBCLOUD-697: allow string for ex_boot_disk on 
create_node

Closes #520

Signed-off-by: Eric Johnson erjoh...@google.com


 GCE Compute driver create_node does not accept str for ex_boot_disk
 ---

 Key: LIBCLOUD-697
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-697
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 The documentation for the GCE compute driver states that the ex_boot_disk 
 param for create_node takes either a StorageVolume or str.  If you pass a 
 string, it results in the following traceback:
 {code:none}
 failed: [localhost] = {failed: true, parsed: false}
 Traceback (most recent call last):
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 2278, in module
 main()
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 577, in main
 module.exit_json(**(node_mgr.execute()))
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 352, in execute
 metadata)
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 461, in create_instances_by_names
 lc_network, lc_zone, metadata)
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 433, in create_instance_by_name
 ex_nic_gce_struct=nics
   File /usr/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 2202, in create_node
 ex_automatic_restart)
   File /usr/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 4507, in _create_node_req
 'deviceName': boot_disk.name,
 AttributeError: 'str' object has no attribute 'name'
 {code}
 The issue appears to be that create_node is passing the ex_boot_disk argument 
 directly to _create_node_req (where the boot_disk param expects a 
 StorageVolume or None).
 Either the docs need to be updated or create_node should instantiate a 
 StorageVolume from ex_boot_disk if it is a str.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] libcloud pull request: [LIBCLOUD-711] Fixed occasional CRC check f...

2015-05-13 Thread chrisob
GitHub user chrisob opened a pull request:

https://github.com/apache/libcloud/pull/519

[LIBCLOUD-711] Fixed occasional CRC check failure when decompressing …

…large responses

fixes issue: https://issues.apache.org/jira/browse/LIBCLOUD-711

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chrisob/libcloud 
LIBCLOUD-711_gzip_crc_check_fail_fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/519.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #519


commit e931f2b9fd19a1d2b16e25ee44c96acb768371dc
Author: Chris O'Brien chriso...@gmail.com
Date:   2015-05-13T19:07:50Z

[LIBCLOUD-711] Fixed occasional CRC check failure when decompressing large 
responses




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (LIBCLOUD-699) GCE Compute driver cannot accept ex_nic_gce_struct for create_node

2015-05-13 Thread Eric Johnson (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542524#comment-14542524
 ] 

Eric Johnson commented on LIBCLOUD-699:
---

Forgot to close this via commit message. Committed upstream 
https://git-wip-us.apache.org/repos/asf?p=libcloud.git;a=commit;h=5c1a95bf199094008c682d3cffab560095668ceb

 GCE Compute driver cannot accept ex_nic_gce_struct for create_node
 --

 Key: LIBCLOUD-699
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-699
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 If you specify the {{ex_nic_gce_struct}} param for the {{create_node}} or 
 {{ex_create_multiple_nodes}}
 Any value other than {{None}} and the test for {{if network and 
 ex_nic_gce_struct:}} in {{_create_node_req}} fails due to network having a 
 default value of 'default'.
 As a workaround I tried to set network to '', since that would have caused it 
 to pass the comparison test, however that leads to an error from the 
 following snippet:
 {code}
 if not hasattr(ex_network, 'name'):
 ex_network = self.ex_get_network(ex_network)
 {code}
 Which returns with the following error:
 {code}
 error: {'locationType': 'parameter', 'domain': 'global', 'message': Invalid 
 value 'None'. Values must match the following regular expression: 
 '[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?', 'reason': 'invalidParameter', 
 'location': 'network'}
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] libcloud pull request: [google compute] fix network options for cr...

2015-05-13 Thread erjohnso
Github user erjohnso closed the pull request at:

https://github.com/apache/libcloud/pull/518


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (LIBCLOUD-711) Periodic GZIP CRC check failure

2015-05-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542472#comment-14542472
 ] 

ASF GitHub Bot commented on LIBCLOUD-711:
-

GitHub user chrisob opened a pull request:

https://github.com/apache/libcloud/pull/519

[LIBCLOUD-711] Fixed occasional CRC check failure when decompressing …

…large responses

fixes issue: https://issues.apache.org/jira/browse/LIBCLOUD-711

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chrisob/libcloud 
LIBCLOUD-711_gzip_crc_check_fail_fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/519.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #519


commit e931f2b9fd19a1d2b16e25ee44c96acb768371dc
Author: Chris O'Brien chriso...@gmail.com
Date:   2015-05-13T19:07:50Z

[LIBCLOUD-711] Fixed occasional CRC check failure when decompressing large 
responses




 Periodic GZIP CRC check failure
 ---

 Key: LIBCLOUD-711
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-711
 Project: Libcloud
  Issue Type: Bug
 Environment: Python 2.6.6
Reporter: Chris O'Brien

 When attempting to parse a gzipped server response, occasionally a CRC check 
 fails while decompressing the response body.  Most of the time the response 
 is correctly decompressed and parsed.  
 Although the compressed data is complete (verified by writing data to file 
 and gunzipping it), this issue only seems to happen with chunked responses.  
 I believe this is due to the fact that the response body is unusually large 
 (~43K uncompressed).  
 I'm using the CloudSigma driver, and this specifically happens with the 
 list_nodes method (the one which responds with the largest amount of data):
 {noformat}
 IOError: CRC check failed 0xdebd5ac != 0x42c31c02L
 ...
 File /home/dbs_support/dev/libcloudsigma/test.py, line 298, in _get_node
   nodes = self.cloud_driver.list_nodes()
 File /home/dbs_support/lib/libcloud/compute/drivers/cloudsigma.py, line 
 1025, in list_nodes
   response = self.connection.request(action=action, method='GET').object
 File /home/dbs_support/lib/libcloud/compute/drivers/cloudsigma.py, line 
 965, in request
   raw=raw)
 File /home/dbs_support/lib/libcloud/common/base.py, line 750, in request
   'response': self.connection.getresponse()}
 File /home/dbs_support/lib/libcloud/common/base.py, line 404, in getresponse
   r, rv = self._log_response(r)
 File /home/dbs_support/lib/libcloud/common/base.py, line 311, in 
 _log_response
   body = decompress_data('gzip', body)
 File /home/dbs_support/lib/libcloud/utils/compression.py, line 39, in 
 decompress_data
   return gzip.GzipFile(fileobj=cls(data)).read()
 File /usr/lib64/python2.6/gzip.py, line 212, in read
   self._read(readsize)
 File /usr/lib64/python2.6/gzip.py, line 267, in _read
   self._read_eof()
 File /usr/lib64/python2.6/gzip.py, line 304, in _read_eof
   hex(self.crc)))
 {noformat}
 In utils/compression.py, would it be wise to replace line #39 ({{return 
 gzip.GzipFile(fileobj=cls(data)).read()}}) to use zlib?  This seems to fix 
 the problem for me, but I'm unaware if there are any negative impacts.  
 For example:
 {noformat}
 decomp = zlib.decompressobj(16+zlib.MAX_WBITS)
 return decomp.decompress(data)
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (LIBCLOUD-697) GCE Compute driver create_node does not accept str for ex_boot_disk

2015-05-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542591#comment-14542591
 ] 

ASF GitHub Bot commented on LIBCLOUD-697:
-

Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/520


 GCE Compute driver create_node does not accept str for ex_boot_disk
 ---

 Key: LIBCLOUD-697
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-697
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 The documentation for the GCE compute driver states that the ex_boot_disk 
 param for create_node takes either a StorageVolume or str.  If you pass a 
 string, it results in the following traceback:
 {code:none}
 failed: [localhost] = {failed: true, parsed: false}
 Traceback (most recent call last):
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 2278, in module
 main()
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 577, in main
 module.exit_json(**(node_mgr.execute()))
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 352, in execute
 metadata)
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 461, in create_instances_by_names
 lc_network, lc_zone, metadata)
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 433, in create_instance_by_name
 ex_nic_gce_struct=nics
   File /usr/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 2202, in create_node
 ex_automatic_restart)
   File /usr/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 4507, in _create_node_req
 'deviceName': boot_disk.name,
 AttributeError: 'str' object has no attribute 'name'
 {code}
 The issue appears to be that create_node is passing the ex_boot_disk argument 
 directly to _create_node_req (where the boot_disk param expects a 
 StorageVolume or None).
 Either the docs need to be updated or create_node should instantiate a 
 StorageVolume from ex_boot_disk if it is a str.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (LIBCLOUD-696) GCE Compute driver returns null for image.

2015-05-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542826#comment-14542826
 ] 

ASF GitHub Bot commented on LIBCLOUD-696:
-

Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/522


 GCE Compute driver returns null for image.
 --

 Key: LIBCLOUD-696
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-696
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 The GCE Compute driver is currently returning null for image.  The issue 
 appears to be related to a change in the data returned from the api.
 In libcloud/compute/drivers/gce.py:
 {code:none|title=_to_node}
 extra['image'] = node.get('image')
 ...
 for disk in extra['disks']:
 if disk.get('boot') and disk.get('type') == 'PERSISTENT':
 bd = self._get_components_from_path(disk['source'])
 extra['boot_disk'] = self.ex_get_volume(bd['name'], bd['zone'])
 extra['image'] = bd['name']
 ...
 if extra['image']:
 image = self._get_components_from_path(extra['image'])['name']
 else:
 image = None
 {code}
 This looks like it should be:
 {code:none}
 for disk in extra['disks']:
 if disk.get('boot') and disk.get('type') == 'PERSISTENT':
 bd = self._get_components_from_path(disk['source'])
 extra['boot_disk'] = self.ex_get_volume(bd['name'], bd['zone'])
 extra['image'] = bd['name']
 ...
 if extra['image']:
 image = self._get_components_from_path(extra['image'])['name']
 else:
 image = None
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


libcloud git commit: [google compute] LIBCLOUD-696: improve node image detection

2015-05-13 Thread erjohnso
Repository: libcloud
Updated Branches:
  refs/heads/trunk aaa8e9e3a - 1e3d3cbc6


[google compute] LIBCLOUD-696: improve node image detection

Closes #522

Signed-off-by: Eric Johnson erjoh...@google.com


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/1e3d3cbc
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/1e3d3cbc
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/1e3d3cbc

Branch: refs/heads/trunk
Commit: 1e3d3cbc6fcc6f3cc862da31b9e17ca5450b95f6
Parents: aaa8e9e
Author: Eric Johnson erjoh...@google.com
Authored: Wed May 13 21:52:39 2015 +
Committer: Eric Johnson erjoh...@google.com
Committed: Wed May 13 22:06:31 2015 +

--
 libcloud/compute/drivers/gce.py | 19 ++-
 1 file changed, 18 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/1e3d3cbc/libcloud/compute/drivers/gce.py
--
diff --git a/libcloud/compute/drivers/gce.py b/libcloud/compute/drivers/gce.py
index 90c1e05..ec5b6f4 100644
--- a/libcloud/compute/drivers/gce.py
+++ b/libcloud/compute/drivers/gce.py
@@ -5262,6 +5262,7 @@ n
 extra['canIpForward'] = node.get('canIpForward')
 extra['serviceAccounts'] = node.get('serviceAccounts', [])
 extra['scheduling'] = node.get('scheduling', {})
+extra['boot_disk'] = None
 
 for disk in extra['disks']:
 if disk.get('boot') and disk.get('type') == 'PERSISTENT':
@@ -5281,10 +5282,17 @@ n
 
 # For the node attributes, use just machine and image names, not full
 # paths.  Full paths are available in the extra dict.
+image = None
 if extra['image']:
 image = self._get_components_from_path(extra['image'])['name']
 else:
-image = None
+if extra['boot_disk'] and \
+hasattr(extra['boot_disk'], 'extra') and \
+'sourceImage' in extra['boot_disk'].extra and \
+extra['boot_disk'].extra['sourceImage'] is not None:
+src_image = extra['boot_disk'].extra['sourceImage']
+image = self._get_components_from_path(src_image)['name']
+extra['image'] = image
 size = self._get_components_from_path(node['machineType'])['name']
 
 return Node(id=node['id'], name=node['name'],
@@ -5422,6 +5430,15 @@ n
 extra['status'] = volume.get('status')
 extra['creationTimestamp'] = volume.get('creationTimestamp')
 extra['description'] = volume.get('description')
+extra['sourceImage'] = volume.get('sourceImage')
+extra['sourceImageId'] = volume.get('sourceImageId')
+extra['sourceSnapshot'] = volume.get('sourceSnapshot')
+extra['sourceSnapshotId'] = volume.get('sourceSnapshotId')
+extra['options'] = volume.get('options')
+if 'licenses' in volume:
+lic_objs = self._licenses_from_urls(licenses=volume['licenses'])
+extra['licenses'] = lic_objs
+
 extra['type'] = volume.get('type', 'pd-standard').split('/')[-1]
 
 return StorageVolume(id=volume['id'], name=volume['name'],



[GitHub] libcloud pull request: [google compute] LIBCLOUD-696: improve node...

2015-05-13 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/522


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (LIBCLOUD-696) GCE Compute driver returns null for image.

2015-05-13 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542824#comment-14542824
 ] 

ASF subversion and git services commented on LIBCLOUD-696:
--

Commit 1e3d3cbc6fcc6f3cc862da31b9e17ca5450b95f6 in libcloud's branch 
refs/heads/trunk from [~erjohnso]
[ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=1e3d3cb ]

[google compute] LIBCLOUD-696: improve node image detection

Closes #522

Signed-off-by: Eric Johnson erjoh...@google.com


 GCE Compute driver returns null for image.
 --

 Key: LIBCLOUD-696
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-696
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 The GCE Compute driver is currently returning null for image.  The issue 
 appears to be related to a change in the data returned from the api.
 In libcloud/compute/drivers/gce.py:
 {code:none|title=_to_node}
 extra['image'] = node.get('image')
 ...
 for disk in extra['disks']:
 if disk.get('boot') and disk.get('type') == 'PERSISTENT':
 bd = self._get_components_from_path(disk['source'])
 extra['boot_disk'] = self.ex_get_volume(bd['name'], bd['zone'])
 extra['image'] = bd['name']
 ...
 if extra['image']:
 image = self._get_components_from_path(extra['image'])['name']
 else:
 image = None
 {code}
 This looks like it should be:
 {code:none}
 for disk in extra['disks']:
 if disk.get('boot') and disk.get('type') == 'PERSISTENT':
 bd = self._get_components_from_path(disk['source'])
 extra['boot_disk'] = self.ex_get_volume(bd['name'], bd['zone'])
 extra['image'] = bd['name']
 ...
 if extra['image']:
 image = self._get_components_from_path(extra['image'])['name']
 else:
 image = None
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (LIBCLOUD-696) GCE Compute driver returns null for image.

2015-05-13 Thread Eric Johnson (JIRA)

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Johnson resolved LIBCLOUD-696.
---
Resolution: Implemented

 GCE Compute driver returns null for image.
 --

 Key: LIBCLOUD-696
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-696
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 The GCE Compute driver is currently returning null for image.  The issue 
 appears to be related to a change in the data returned from the api.
 In libcloud/compute/drivers/gce.py:
 {code:none|title=_to_node}
 extra['image'] = node.get('image')
 ...
 for disk in extra['disks']:
 if disk.get('boot') and disk.get('type') == 'PERSISTENT':
 bd = self._get_components_from_path(disk['source'])
 extra['boot_disk'] = self.ex_get_volume(bd['name'], bd['zone'])
 extra['image'] = bd['name']
 ...
 if extra['image']:
 image = self._get_components_from_path(extra['image'])['name']
 else:
 image = None
 {code}
 This looks like it should be:
 {code:none}
 for disk in extra['disks']:
 if disk.get('boot') and disk.get('type') == 'PERSISTENT':
 bd = self._get_components_from_path(disk['source'])
 extra['boot_disk'] = self.ex_get_volume(bd['name'], bd['zone'])
 extra['image'] = bd['name']
 ...
 if extra['image']:
 image = self._get_components_from_path(extra['image'])['name']
 else:
 image = None
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] libcloud pull request: [google compute] LIBCLOUD-696: improve node...

2015-05-13 Thread erjohnso
GitHub user erjohnso opened a pull request:

https://github.com/apache/libcloud/pull/522

[google compute] LIBCLOUD-696: improve node image detection

Google removed 'image' from the instance resource a while ago, but 
sometimes set a sourceImage on a persistent disk. This fix will consult the 
sourceImage attribute on the instance's (node's) boot_disk to try and determine 
the image.  There is still no guarantee this will be set (e.g. if a persistent 
disk was created from a snapshot, this will still be None), but this is an 
improvement.

Fixes https://issues.apache.org/jira/browse/LIBCLOUD-696

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/erjohnso/libcloud jira696

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/522.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #522


commit b5fd83947f6fb821c38618782f1de39d73a6e0a4
Author: Eric Johnson erjoh...@google.com
Date:   2015-05-13T21:52:39Z

[google compute] LIBCLOUD-696: improve node image detection




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Updated] (LIBCLOUD-709) Implement retry on rate limit for nova api

2015-05-13 Thread Arpith Prakash (JIRA)

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpith Prakash updated LIBCLOUD-709:

Description: Nova raises exception when there are too many request, and 
retry-after time is being sent within the header. We should be catching that 
exception and retry automatically which could resolve most of downstream 
additional logic of implement it.  (was: Nova client raises exception when 
there are too many request, and retry-after time is being sent within the 
header. We should be catching that exception and retry automatically which 
could resolve most of downstream additional logic of implement it.)

 Implement retry on rate limit for nova api
 --

 Key: LIBCLOUD-709
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-709
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Arpith Prakash
Priority: Trivial

 Nova raises exception when there are too many request, and retry-after time 
 is being sent within the header. We should be catching that exception and 
 retry automatically which could resolve most of downstream additional logic 
 of implement it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] libcloud pull request: [LIBCLOUD-360] [LIBCLOUD-709] Retry on rate...

2015-05-13 Thread arpithkp
GitHub user arpithkp opened a pull request:

https://github.com/apache/libcloud/pull/515

[LIBCLOUD-360] [LIBCLOUD-709] Retry on rate limit and connection timeout.

[Retry Logic](https://issues.apache.org/jira/browse/LIBCLOUD-360)
[Rate Limit Logic](https://issues.apache.org/jira/browse/LIBCLOUD-709)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/arpithkp/libcloud retry_rate_limit

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/515.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #515


commit 38158c8e67d3539fb8aa619adc6b726479153955
Author: Arpith Kuppur Prakash arpith.kuppurprak...@rackspace.com
Date:   2015-05-06T19:02:17Z

[LIBCLOUD-360] [LIBCLOUD-709] Retry on rate limit and connection timeout.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (LIBCLOUD-710) CloudSigma driver doesn't take advantage of new(er) NodeStates

2015-05-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542265#comment-14542265
 ] 

ASF GitHub Bot commented on LIBCLOUD-710:
-

GitHub user chrisob opened a pull request:

https://github.com/apache/libcloud/pull/517

[LIBCLOUD-710] Updated CloudSigma's NODE_STATE_MAP to use new-ish Nod…

…eStates

Fixes issue: https://issues.apache.org/jira/browse/LIBCLOUD-710

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chrisob/libcloud 
LIBCLOUD-710_cloudsigma_update_node_state_map

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/517.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #517


commit 011c6767c40a52588138477f22a8fe11b223da08
Author: Chris O'Brien chriso...@gmail.com
Date:   2015-05-13T17:14:47Z

[LIBCLOUD-710] Updated CloudSigma's NODE_STATE_MAP to use new-ish NodeStates




 CloudSigma driver doesn't take advantage of new(er) NodeStates
 --

 Key: LIBCLOUD-710
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-710
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Chris O'Brien
Priority: Minor

 The NodeStates 'ERROR', and 'PAUSED' are not utilized for the corresponding 
 CloudSigma statuses (in CloudSigma_2_0_NodeDriver's NODE_STATE_MAP) 
 'unavailable' and 'paused'.  
 I propose having CloudSigma's 'unavailable' state translate to 
 NodeState.ERROR and the 'paused' state translate to NodeState.PAUSED 
 (duh...).  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] libcloud pull request: [LIBCLOUD-710] Updated CloudSigma's NODE_ST...

2015-05-13 Thread chrisob
GitHub user chrisob opened a pull request:

https://github.com/apache/libcloud/pull/517

[LIBCLOUD-710] Updated CloudSigma's NODE_STATE_MAP to use new-ish Nod…

…eStates

Fixes issue: https://issues.apache.org/jira/browse/LIBCLOUD-710

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chrisob/libcloud 
LIBCLOUD-710_cloudsigma_update_node_state_map

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/517.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #517


commit 011c6767c40a52588138477f22a8fe11b223da08
Author: Chris O'Brien chriso...@gmail.com
Date:   2015-05-13T17:14:47Z

[LIBCLOUD-710] Updated CloudSigma's NODE_STATE_MAP to use new-ish NodeStates




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (LIBCLOUD-699) GCE Compute driver cannot accept ex_nic_gce_struct for create_node

2015-05-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542367#comment-14542367
 ] 

ASF GitHub Bot commented on LIBCLOUD-699:
-

GitHub user erjohnso opened a pull request:

https://github.com/apache/libcloud/pull/518

[google compute] fix network options for create_node

Minor fix to detect network option conflicts as described in 
https://issues.apache.org/jira/browse/LIBCLOUD-699

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/erjohnso/libcloud jira699

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/518.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #518






 GCE Compute driver cannot accept ex_nic_gce_struct for create_node
 --

 Key: LIBCLOUD-699
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-699
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 If you specify the {{ex_nic_gce_struct}} param for the {{create_node}} or 
 {{ex_create_multiple_nodes}}
 Any value other than {{None}} and the test for {{if network and 
 ex_nic_gce_struct:}} in {{_create_node_req}} fails due to network having a 
 default value of 'default'.
 As a workaround I tried to set network to '', since that would have caused it 
 to pass the comparison test, however that leads to an error from the 
 following snippet:
 {code}
 if not hasattr(ex_network, 'name'):
 ex_network = self.ex_get_network(ex_network)
 {code}
 Which returns with the following error:
 {code}
 error: {'locationType': 'parameter', 'domain': 'global', 'message': Invalid 
 value 'None'. Values must match the following regular expression: 
 '[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?', 'reason': 'invalidParameter', 
 'location': 'network'}
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] libcloud pull request: [google compute] fix network options for cr...

2015-05-13 Thread erjohnso
GitHub user erjohnso opened a pull request:

https://github.com/apache/libcloud/pull/518

[google compute] fix network options for create_node

Minor fix to detect network option conflicts as described in 
https://issues.apache.org/jira/browse/LIBCLOUD-699

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/erjohnso/libcloud jira699

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/518.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #518






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Resolved] (LIBCLOUD-699) GCE Compute driver cannot accept ex_nic_gce_struct for create_node

2015-05-13 Thread Eric Johnson (JIRA)

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Johnson resolved LIBCLOUD-699.
---
Resolution: Implemented

 GCE Compute driver cannot accept ex_nic_gce_struct for create_node
 --

 Key: LIBCLOUD-699
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-699
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 If you specify the {{ex_nic_gce_struct}} param for the {{create_node}} or 
 {{ex_create_multiple_nodes}}
 Any value other than {{None}} and the test for {{if network and 
 ex_nic_gce_struct:}} in {{_create_node_req}} fails due to network having a 
 default value of 'default'.
 As a workaround I tried to set network to '', since that would have caused it 
 to pass the comparison test, however that leads to an error from the 
 following snippet:
 {code}
 if not hasattr(ex_network, 'name'):
 ex_network = self.ex_get_network(ex_network)
 {code}
 Which returns with the following error:
 {code}
 error: {'locationType': 'parameter', 'domain': 'global', 'message': Invalid 
 value 'None'. Values must match the following regular expression: 
 '[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?', 'reason': 'invalidParameter', 
 'location': 'network'}
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (LIBCLOUD-697) GCE Compute driver create_node does not accept str for ex_boot_disk

2015-05-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542580#comment-14542580
 ] 

ASF GitHub Bot commented on LIBCLOUD-697:
-

GitHub user erjohnso opened a pull request:

https://github.com/apache/libcloud/pull/520

[google compute] bugfix: allow string for ex_boot_disk on create_node

Bugfix to allow a `str` argument for `ex_boot_disk` on `create_node()`.  
This is a fix for https://issues.apache.org/jira/browse/LIBCLOUD-697

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/erjohnso/libcloud jira697

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/520.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #520


commit 69ec7aa0bae33ad77d668b7c10e123f8a30e1e83
Author: Eric Johnson erjoh...@google.com
Date:   2015-05-13T19:53:32Z

[google compute] bugfix: allow string for ex_boot_disk on create_node




 GCE Compute driver create_node does not accept str for ex_boot_disk
 ---

 Key: LIBCLOUD-697
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-697
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Jason DeTiberus
Assignee: Eric Johnson

 The documentation for the GCE compute driver states that the ex_boot_disk 
 param for create_node takes either a StorageVolume or str.  If you pass a 
 string, it results in the following traceback:
 {code:none}
 failed: [localhost] = {failed: true, parsed: false}
 Traceback (most recent call last):
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 2278, in module
 main()
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 577, in main
 module.exit_json(**(node_mgr.execute()))
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 352, in execute
 metadata)
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 461, in create_instances_by_names
 lc_network, lc_zone, metadata)
   File 
 /home/jdetiber/.ansible/tmp/ansible-tmp-1429541392.81-113800638314850/gce, 
 line 433, in create_instance_by_name
 ex_nic_gce_struct=nics
   File /usr/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 2202, in create_node
 ex_automatic_restart)
   File /usr/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 4507, in _create_node_req
 'deviceName': boot_disk.name,
 AttributeError: 'str' object has no attribute 'name'
 {code}
 The issue appears to be that create_node is passing the ex_boot_disk argument 
 directly to _create_node_req (where the boot_disk param expects a 
 StorageVolume or None).
 Either the docs need to be updated or create_node should instantiate a 
 StorageVolume from ex_boot_disk if it is a str.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] libcloud pull request: [google compute] bugfix: allow string for e...

2015-05-13 Thread erjohnso
GitHub user erjohnso opened a pull request:

https://github.com/apache/libcloud/pull/520

[google compute] bugfix: allow string for ex_boot_disk on create_node

Bugfix to allow a `str` argument for `ex_boot_disk` on `create_node()`.  
This is a fix for https://issues.apache.org/jira/browse/LIBCLOUD-697

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/erjohnso/libcloud jira697

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/520.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #520


commit 69ec7aa0bae33ad77d668b7c10e123f8a30e1e83
Author: Eric Johnson erjoh...@google.com
Date:   2015-05-13T19:53:32Z

[google compute] bugfix: allow string for ex_boot_disk on create_node




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


libcloud git commit: [google compute] fixes LIBCLOUD-697: allow string for ex_boot_disk on create_node

2015-05-13 Thread erjohnso
Repository: libcloud
Updated Branches:
  refs/heads/trunk 5c1a95bf1 - 8664118c8


[google compute] fixes LIBCLOUD-697: allow string for ex_boot_disk on 
create_node

Closes #520

Signed-off-by: Eric Johnson erjoh...@google.com


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/8664118c
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/8664118c
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/8664118c

Branch: refs/heads/trunk
Commit: 8664118c877a43f1434a576bd46223707589b078
Parents: 5c1a95b
Author: Eric Johnson erjoh...@google.com
Authored: Wed May 13 19:53:32 2015 +
Committer: Eric Johnson erjoh...@google.com
Committed: Wed May 13 20:01:46 2015 +

--
 libcloud/compute/drivers/gce.py | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/8664118c/libcloud/compute/drivers/gce.py
--
diff --git a/libcloud/compute/drivers/gce.py b/libcloud/compute/drivers/gce.py
index 22f3f9a..4f15715 100644
--- a/libcloud/compute/drivers/gce.py
+++ b/libcloud/compute/drivers/gce.py
@@ -2253,7 +2253,7 @@ class GCENodeDriver(NodeDriver):
 :type ex_metadata: ``dict`` or ``None``
 
 :keyword  ex_boot_disk: The boot disk to attach to the instance.
-:type ex_boot_disk: :class:`StorageVolume` or ``str``
+:type ex_boot_disk: :class:`StorageVolume` or ``str`` or ``None``
 
 :keyword  use_existing_disk: If True and if an existing disk with the
  same name/location is found, use that
@@ -2265,7 +2265,7 @@ class GCENodeDriver(NodeDriver):
used.  If 'None', then no external address will
be used.  To use an existing static IP address,
a GCEAddress object should be passed in.
-:type external_ip: :class:`GCEAddress` or ``str`` or None
+:type external_ip: :class:`GCEAddress` or ``str`` or ``None``
 
 :keyword  ex_disk_type: Specify a pd-standard (default) disk or pd-ssd
 for an SSD disk.
@@ -2356,6 +2356,8 @@ n
 image = self.ex_get_image(image)
 if not hasattr(ex_disk_type, 'name'):
 ex_disk_type = self.ex_get_disktype(ex_disk_type, zone=location)
+if ex_boot_disk and not hasattr(ex_boot_disk, 'name'):
+ex_boot_disk = self.ex_get_disk(ex_boot_disk, zone=location)
 
 # Use disks[].initializeParams to auto-create the boot disk
 if not ex_disks_gce_struct and not ex_boot_disk:
@@ -4601,7 +4603,7 @@ n
is made to ensure proper formatting of
the disks[] structure. Using this
structure obviates the need of using
-   other disk params like 'ex_boot_disk',
+   other disk params like 'boot_disk',
etc. See the GCE docs for specific
details.
 :type ex_disks_gce_struct: ``list`` or ``None``
@@ -4840,7 +4842,7 @@ n
 request, node_data = self._create_node_req(
 status['name'], node_attrs['size'], node_attrs['image'],
 node_attrs['location'], node_attrs['network'], node_attrs['tags'],
-node_attrs['metadata'], boot_disk=status['disk'],
+node_attrs['metadata'],
 external_ip=node_attrs['external_ip'],
 ex_service_accounts=node_attrs['ex_service_accounts'],
 description=node_attrs['description'],



[jira] [Resolved] (LIBCLOUD-687) Unable to set GCE metadata if no metadata already exists

2015-05-13 Thread Eric Johnson (JIRA)

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Johnson resolved LIBCLOUD-687.
---
Resolution: Implemented

 Unable to set GCE metadata if no metadata already exists
 

 Key: LIBCLOUD-687
 URL: https://issues.apache.org/jira/browse/LIBCLOUD-687
 Project: Libcloud
  Issue Type: Bug
  Components: Compute
Reporter: Eli Courtwright
Assignee: Eric Johnson

 The {{libcloud.compute.drivers.gce.GCENodeDriver}} class has a nonstandard 
 method {{ex_set_common_metadata}} which we can use to (among other things) 
 set SSH keys.
 When calling this method on the latest libcloud 0.17.0 (which is not 
 currently a listed version in Jira), I get the following error:
 {code}
  conn.ex_set_common_instance_metadata({})
 Traceback (most recent call last):
   File stdin, line 1, in module
   File 
 /opt/sideboard/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py, 
 line 1129, in ex_set_common_instance_metadata
 for md in current_metadata['items']:
 KeyError: 'items'
 {code}
 I get this error regardless of what data I pass to the 
 {{ex_set_common_instance_metadata}} method.  From delving into the libcloud 
 code, I find the following block of code:
 {code}
 project = self.ex_get_project()
 current_metadata = project.extra['commonInstanceMetadata']
 fingerprint = current_metadata['fingerprint']
 # grab copy of current 'sshKeys' in case we want to retain them
 current_keys = 
 for md in current_metadata['items']:
 {code}
 So it looks as if the method assumes that {{self.ex_get_project()}} will 
 return:
 - an object with a {{.extra}} dictionary (which it does)
 - that dictionary will have a commonInstanceMetadata key which maps to 
 another dictionary (which is correct)
 - that sub-dictionary will have an items key whose value is iterable (which 
 is wrong)
 That last assumption is only wrong if there is no project metadata already 
 associated with the project, e.g. for a new project.
 I'm working around this problem currently by monkeypatching libcloud:
 {code}
 def patch_gce_ex_get_project():
 orig_get_project = GCENodeDriver.ex_get_project
 def ex_get_project(self):
 project = orig_get_project(self)
 project.extra['commonInstanceMetadata'].setdefault('items', [])
 return project
 GCENodeDriver.ex_get_project = ex_get_project
 {code}
 but having a real fix would be great for the next version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)