That's the best workaround I've got at this point, yeah, but I'd really like to find a good way of doing bulk polling - I don't understand LoadingCache/CacheLoader well enough, but was thinking maybe something along the lines of having RefreshNodeAndDoubleCheckOnFailUnlessStatusInvalid check a cache, where that cache expires entries after the polling period has passed, and when the cache expires (or it needs to load for the first time), it can do a bulk get nodes query (if available) for all the nodes it needs to know about...but I'm not sure how to be sure it knows all the relevant nodes, how to keep it from slamming with a bunch of bulk queries if you're creating nodes consecutively, etc...
I'll play with this, in the context of JCLOUDS-450 (which I just opened)... A. On Tue, Feb 4, 2014 at 6:46 AM, Everett Toews <[email protected]>wrote: > On Feb 3, 2014, at 6:28 PM, Andrew Bayer <[email protected]> wrote: > > > So I'm trying to figure out how to make it possible to provision 100+ > > instances on EC2 in one createNodesInGroup call without getting crushed > by > > API rate limits. The combination of the min count/max count parameters > and > > the idempotency with a client token (see > > > http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Run_Instance_Idempotency.html > ) > > makes the raw creation of 100+ instances without a thundering herd of > > RunInstances calls very realistic, but then we hit the issue of polling > > those nodes (even with the idempotency, we'd still end up having at least > > one is-node-running polling call per node, and then we'd hit the same > thing > > but much bigger at terminate/pause/resume/etc)... > > The way I've dealt with this in the past is by using POLL_INITIAL_PERIOD > and POLL_MAX_PERIOD [1]. > > Does that work for your use case? > > Everett > > [1] > https://github.com/jclouds/jclouds-examples/blob/master/rackspace/src/main/java/org/jclouds/examples/rackspace/cloudservers/CloudServersPublish.java#L102-103
