My apologies to Ignasi Barrera who already replied to this; but I
apparently had some issues being subscribed to the list and don't have
his message to reply to, so I'm replying to my own message to maintain
threading.
I did manage to get my multithreaded server creation working, and it was
my own stupidity at fault.
I had instantiated a single computeService object prior to the various
threaded context.getComputeService() calls, and this meant they were all
sharing (and overwriting references to) the same computeService object.
As soon as one of the threads finished, one of the two possible errors
would occur depending on if the first thread was the one which was not
already null. Easily resolved by giving each thread its own
computeService instantiation.
On 5/6/14, 3:42 PM, Daniel Widdis wrote:
I'm relatively new to jclouds, using it to create (and delete) servers
on Rackspace, and have been steadily trying to improve its use in my code.
I have developed a class based on the excellent examples provided in
org.jclouds.examples.rackspace.cloudservers and they have been working
fine.
In my code I have a need to build three types of servers from my saved
images, and am currently doing so with consecutive calls to a server
creation class, which eventually calls
computeService.createNodesInGroup(). The build process has been
taking from a few seconds to a few minutes routinely, with occasional
long creation times of about 20 minutes, but usually in about the 3 or
4 minute range. Building three different types of images in sequence
has been taking up to 10 minutes to complete.
In order to try to speed up this process, I have tried to multi-thread
my server generation calls using the Callable interface (basically
replacing create() and delete() with call() and using an
ExecutorService to submit() the creation task and get() the resulting
NodeMetadata. The code executes just fine in a test case where I
invoke the call() method sequentially, but when I submit the same code
to the Executor, it does not work. I am successfully creating all the
servers I ask for, but the threaded calls I'm using to return the
NodeMetadata are all timing out with errors similar to this:
SEVERE: << problem customizing
node(IAD/a04b5524-1c75-459b-b7ec-bd3256f52928):
java.lang.IllegalStateException:
node(IAD/a04b5524-1c75-459b-b7ec-bd3256f52928) didn't achieve the
status running; aborting after 124 seconds with final status: PENDING
I've seen times from 124 seconds (shortest) to 179 seconds (longest).
After research, I found jclouds.compute.timeout.node-running value in
the org.jclouds.compute.functions.PollNodeRunning class is what I
wanted to try to manipulate here, and used ComputeService Properties
to override that setting with 30 minutes; however, that introduces a
new error:
SEVERE: RuntimeException while executing runnable
callGetOnFuture(server-test-5be,com.google.common.util.concurrent.Futures$ChainingListenableFuture@2a6a4239)
with executor
com.google.common.util.concurrent.MoreExecutors$ListeningDecorator@2621c1f0
java.util.concurrent.RejectedExecutionException:
java.lang.InterruptedException
at
org.jclouds.concurrent.DynamicThreadPoolExecutor$ForceQueuePolicy.rejectedExecution(DynamicThreadPoolExecutor.java:132)
So, my questions are:
1. Why is the timeout.node-running value different when running in a
Callable thread vs. when running sequentially? What is its default?
2. Why might I be getting the second exception? Is it not possible to
multithread server creation? Are there any working examples of a
multithreaded implementation?
