Hello Folks,
I just deployed a Cloudstack 4.17.2 with VMware 7.0. I am able to provision
system VMs and VRs but not able to provision VMs.
While checking the logs - looks like it’s not able to connect to vCenter and
not able to create disks, but somehow VRs and system VMs are being provisioned
perfectly fine.
———————————————————
In management servers logs
———————————————————
2023-11-27 18:27:25,688 DEBUG [c.c.a.t.Request] (AgentManager-Handler-3:null)
(logid:) Seq 11-6334875825850024098: Processing: { Ans: , MgmtId:
7115655717170, via: 11, Ver: v1, Flags: 110,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":"false","details":"Failed
to create a VMware context, check the management server logs or the SSVM log
for details","wait":"0","bypassHostMaintenance":"false"}}] }
2023-11-27 18:27:25,688 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-5:ctx-164cb640 job-94/job-95 ctx-8ed050e5) (logid:09eb73ef)
Seq 11-6334875825850024098: Received: { Ans: , MgmtId: 7115655717170, via:
11(s-8-VM), Ver: v1, Flags: 110, { CopyCmdAnswer } }
2023-11-27 18:27:25,717 DEBUG [c.c.a.m.AgentAttache]
(AgentManager-Handler-3:null) (logid:) Seq 11-6334875825850024098: No more
commands found
2023-11-27 18:27:25,740 DEBUG [o.a.c.h.HAManagerImpl]
(BackgroundTaskPollManager-5:ctx-4f6c8aaa) (logid:03ac36da) HA health check
task is running...
2023-11-27 18:27:25,743 INFO [o.a.c.s.v.VolumeServiceImpl]
(Work-Job-Executor-5:ctx-164cb640 job-94/job-95 ctx-8ed050e5) (logid:09eb73ef)
releasing lock for VMTemplateStoragePool 8
2023-11-27 18:27:25,751 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-5:ctx-164cb640 job-94/job-95 ctx-8ed050e5) (logid:09eb73ef)
Unable to create Vol[15|vm=15|ROOT]:Failed to create a VMware context, check
the management server logs or the SSVM log for details
2023-11-27 18:27:25,752 WARN [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-5:ctx-164cb640 job-94/job-95 ctx-8ed050e5) (logid:09eb73ef)
Unable to contact resource.
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:3] is
unreachable: Unable to create Vol[15|vm=15|ROOT]:Failed to create a VMware
context, check the management server logs or the SSVM log for details
at
org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.recreateVolume(VolumeOrchestrator.java:1595)
at
org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.prepare(VolumeOrchestrator.java:1689)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1179)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5315)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5439)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2023-11-27 18:27:25,763 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-5:ctx-164cb640 job-94/job-95 ctx-8ed050e5) (logid:09eb73ef)
Cleaning up resources for the vm VM instance {id: "15", name: "i-2-15-VM",
uuid: "2b78b7af-d4c6-428a-b3f0-f7f99995760b", type="User"} in Starting state
———————————————————
In SSVM logs
———————————————————
2023-11-27 15:32:40,663 INFO
[storage.resource.VmwareSecondaryStorageResourceHandler]
(agentRequest-Handler-2:job-98/job-100, cmd: CopyCommand) Open new
VmwareContext. vCenter: X.X.X.xX, user: [email protected], password:
v**********
2023-11-27 15:33:19,030 INFO [storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-4:null) Determined host 10.60.6.51 corresponds to IP
10.60.6.51
2023-11-27 15:33:19,238 INFO [storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-3:null) Determined host 10.60.6.51 corresponds to IP
10.60.6.51
2023-11-27 15:34:19,297 INFO [storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-1:null) Determined host 10.60.6.51 corresponds to IP
10.60.6.51
2023-11-27 15:34:19,514 INFO [storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-5:null) Determined host 10.60.6.51 corresponds to IP
10.60.6.51
2023-11-27 15:34:50,070 ERROR
[storage.resource.VmwareSecondaryStorageResourceHandler]
(agentRequest-Handler-2:job-98/job-100, cmd: CopyCommand) Unexpected exception
com.sun.xml.ws.client.ClientTransportException: HTTP transport error:
java.net.ConnectException: Connection timed out (Connection timed out)
com.sun.xml.ws.client.ClientTransportException: HTTP transport error:
java.net.ConnectException: Connection timed out (Connection timed out)
at
com.sun.xml.ws.transport.http.client.HttpClientTransport.getOutput(HttpClientTransport.java:102)
at
com.sun.xml.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:193)
at
com.sun.xml.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:115)
at
com.sun.xml.ws.transport.DeferredTransportPipe.processRequest(DeferredTransportPipe.java:109)
at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:1106)
at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:1020)
at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:989)
at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:847)
at com.sun.xml.ws.client.Stub.process(Stub.java:433)
at com.sun.xml.ws.client.sei.SEIStub.doProcess(SEIStub.java:161)
at
com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:78)
at
com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:62)
at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:131)
at com.sun.proxy.$Proxy46.retrieveServiceContent(Unknown Source)
at
com.cloud.hypervisor.vmware.util.VmwareClient.connect(VmwareClient.java:174)
at
com.cloud.storage.resource.VmwareSecondaryStorageContextFactory.create(VmwareSecondaryStorageContextFactory.java:47)
at
com.cloud.storage.resource.VmwareSecondaryStorageContextFactory.getContext(VmwareSecondaryStorageContextFactory.java:59)
at
com.cloud.storage.resource.VmwareSecondaryStorageResourceHandler.getServiceContext(VmwareSecondaryStorageResourceHandler.java:228)
at
com.cloud.storage.resource.VmwareStorageProcessor.copyTemplateToPrimaryStorage(VmwareStorageProcessor.java:633)
at
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:97)
at
com.cloud.storage.resource.VmwareStorageSubsystemCommandHandler.execute(VmwareStorageSubsystemCommandHandler.java:201)
at
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:61)
at
com.cloud.storage.resource.VmwareSecondaryStorageResourceHandler.executeRequest(VmwareSecondaryStorageResourceHandler.java:113)
at
com.cloud.storage.resource.PremiumSecondaryStorageResource.executeRequest(PremiumSecondaryStorageResource.java:57)
at com.cloud.agent.Agent.processRequest(Agent.java:660)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1080)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.net.ConnectException: Connection timed out (Connection timed
out)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
at
java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
at
java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/java.net.Socket.connect(Socket.java:558)
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:182)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:508)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:603)
at
java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:266)
at
java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:373)
at
java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:207)
at
java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1232)
at
java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
at
java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:193)
at
java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1367)
at
java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1342)
at
java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:246)
at
com.sun.xml.ws.transport.http.client.HttpClientTransport.getOutput(HttpClientTransport.java:89)
... 31 more
Vivek Kumar
Sr. Manager - Cloud & DevOps
TechOps | Indiqus Technologies
[email protected] <mailto:[email protected]>
www.indiqus.com <https://www.indiqus.com/>
--
This message is intended only for the use of the individual or entity to
which it is addressed and may contain confidential and/or privileged
information. If you are not the intended recipient, please delete the
original message and any copy of it from your computer system. You are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited unless proper authorization has been
obtained for such action. If you have received this communication in error,
please notify the sender immediately. Although IndiQus attempts to sweep
e-mail and attachments for viruses, it does not guarantee that both are
virus-free and accepts no liability for any damage sustained as a result of
viruses.
