Hello: I recently added support for Libcloud to our internal infrastructure management tool as part of a project to support GCP (and other providers). We are currently managing a deployment of 50 or so instances which we plan to grow to several hundred.
In order to optimize costs, we have several custom threads which monitor usage and power up/down instances when they are needed. This is managed with several Python threads which are running in parallel. Most of the application state is kept locally to reduce API calls, but there are a few things that we rely on the cloud API to provide us, in particular the instance state (running/stopped/etc). The issue we ran into is that we restarted these threads and they all went into a cycle of calling ex_get_node() for each instance to refresh the state and other instance metadata. This quickly swamped our API quota for GCP, specifically the HeavyWeightReadGroup request. We have since made some changes to avoid this scenario and to reduce the number of calls we make (and we got our quota increased), but as we eye bigger deployments I'm wondering if there is a better way? If I only need to refresh the instance.state, is there a different API call I can make? I couldn't find anything other than to rebuild my local object with a call to ex_get_node(). For example, boto's EC2 Instance class has an update() method which refreshes the state and tags - Is there something similar in Libcloud? Thanks, -poul
