GitHub user Ahuri3 opened a pull request:
https://github.com/apache/libcloud/pull/1203
Fixed bug where the method _find_zone_or_region cannot see some instances
## Fixed bug where _find_zone_or_region cannot see some instances
### Description
The method `_find_zone_or_region` queries the google cloud api on the
endpoint `instances.aggregatedList`.
This endpoint cannot return more than 500 results but has a pagination
mechanic. The method `_find_zone_or_region` does not make use of the pagination
mechanic because it made a manual request to the api.
I removed the manual request and replaced it by the method
`request_aggregated_items` that was made for this exact purpose.
This bug was found while trying to use the libcloud in ansible (via the
gce_snapshot ansible module) and getting an error
`libcloud.common.google.ResourceNotFoundError: "Node 'node_name' not found in
any zone."`.
Indeed without the pagination, if a google cloud project has more than 500
instances, the code will crash when trying to identify the zone for a gce
instance that wasn't in the 500 instances returned by the first api call.
Here is the stack trace I saw :
```
The full traceback is:
Traceback (most recent call last):
File "/tmp/ansible_qBlsq5/ansible_module_gce_snapshot.py", line 225, in
<module>
main()
File "/tmp/ansible_qBlsq5/ansible_module_gce_snapshot.py", line 190, in
main
instance = gce.ex_get_node(instance_name, 'all')
File
"/var/lib/rundeck/.conda/envs/eva-prod/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py",
line 7030, in ex_get_node
name, 'instances', res_name='Node')
File
"/var/lib/rundeck/.conda/envs/eva-prod/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py",
line 7561, in _find_zone_or_region
(res_name, name, rz), None, None)
libcloud.common.google.ResourceNotFoundError: "Node 'pr-op-cass00-0004' not
found in any zone."
fatal: [pr-op-cass00-0004 -> localhost]: FAILED! => {
"changed": false,
"module_stderr": "Traceback (most recent call last):\n File
\"/tmp/ansible_Gk7vXV/ansible_module_gce_snapshot.py\", line 225, in <module>\n
main()\n File \"/tmp/ansible_Gk7vXV/ansible_module_gce_snapshot.py\", line
190, in main\n instance = gce.ex_get_node(instance_name, 'all')\n File
\"/var/lib/rundeck/.conda/envs/eva-prod/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py\",
line 7030, in ex_get_node\n name, 'instances', res_name='Node')\n File
\"/var/lib/rundeck/.conda/envs/eva-prod/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py\",
line 7561, in _find_zone_or_region\n (res_name, name, rz), None,
None)\nlibcloud.common.google.ResourceNotFoundError: \"Node 'pr-op-cass00-0004'
not found in any zone.\"\n",
"module_stdout": "",
"msg": "MODULE FAILURE",
"rc": 1
}
...ignoring
```
### Status
ready for review
### Checklist (tick everything that applies)
- [x] [Code
linting](http://libcloud.readthedocs.org/en/latest/development.html#code-style-guide)
(required, can be done after the PR checks)
- No pagination was changed
- Line lenght is shorter than 79 characters
- [ ] Documentation
- N/A
- [x] [Tests](http://libcloud.readthedocs.org/en/latest/testing.html)
- Tested with `Python 2.7.13 :: Continuum Analytics, Inc.`
- [ ]
[ICLA](http://libcloud.readthedocs.org/en/latest/development.html#contributing-bigger-changes)
(required for bigger changes)
- N/A
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Ahuri3/libcloud
fix_find_zone_or_region_by_using_request_aggregated_items
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/libcloud/pull/1203.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1203
----
commit 4a7d884ca6e9d1864320bf3b21087fb66ac21d01
Author: Léo Ferlin-Sutton <lferlin@...>
Date: 2018-04-26T13:22:17Z
Modified the way '_find_zone_or_region' gets data from the
'instances.aggregatedList' api by using the method 'request_aggregated_items'
which correctly paginates
----
---
