I'm relatively new to jclouds, using it to create (and delete) servers
on Rackspace, and have been steadily trying to improve its use in my code.
I have developed a class based on the excellent examples provided in
org.jclouds.examples.rackspace.cloudservers and they have been working
fine.
In my code I have a need to build three types of servers from my saved
images, and am currently doing so with consecutive calls to a server
creation class, which eventually calls
computeService.createNodesInGroup(). The build process has been taking
from a few seconds to a few minutes routinely, with occasional long
creation times of about 20 minutes, but usually in about the 3 or 4
minute range. Building three different types of images in sequence has
been taking up to 10 minutes to complete.
In order to try to speed up this process, I have tried to multi-thread
my server generation calls using the Callable interface (basically
replacing create() and delete() with call() and using an ExecutorService
to submit() the creation task and get() the resulting NodeMetadata.
The code executes just fine in a test case where I invoke the call()
method sequentially, but when I submit the same code to the Executor, it
does not work. I am successfully creating all the servers I ask for, but
the threaded calls I'm using to return the NodeMetadata are all timing
out with errors similar to this:
SEVERE: << problem customizing
node(IAD/a04b5524-1c75-459b-b7ec-bd3256f52928):
java.lang.IllegalStateException:
node(IAD/a04b5524-1c75-459b-b7ec-bd3256f52928) didn't achieve the status
running; aborting after 124 seconds with final status: PENDING
I've seen times from 124 seconds (shortest) to 179 seconds (longest).
After research, I found jclouds.compute.timeout.node-running value in
the org.jclouds.compute.functions.PollNodeRunning class is what I wanted
to try to manipulate here, and used ComputeService Properties to
override that setting with 30 minutes; however, that introduces a new error:
SEVERE: RuntimeException while executing runnable
callGetOnFuture(server-test-5be,com.google.common.util.concurrent.Futures$ChainingListenableFuture@2a6a4239)
with executor
com.google.common.util.concurrent.MoreExecutors$ListeningDecorator@2621c1f0
java.util.concurrent.RejectedExecutionException:
java.lang.InterruptedException
at
org.jclouds.concurrent.DynamicThreadPoolExecutor$ForceQueuePolicy.rejectedExecution(DynamicThreadPoolExecutor.java:132)
So, my questions are:
1. Why is the timeout.node-running value different when running in a
Callable thread vs. when running sequentially? What is its default?
2. Why might I be getting the second exception? Is it not possible to
multithread server creation? Are there any working examples of a
multithreaded implementation?
