Hi Sanjay, Thanks for the log snippet, would you be able to share the logs from before this point. The actual reason of failure may be present in the earlier bits of the log.
Thanks. ________________________________ From: Sanjay Kumar <[email protected]> Sent: December 5, 2025 7:02 AM To: [email protected] <[email protected]> Cc: [email protected] <[email protected]> Subject: Re: We are facing issue while using multi-ARC in same zone. Hi Pearl, Thanks for the quick response. 2025-12-05 01:28:45,000 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-e53fd38f]) (logid:984042af) Peer scan is finished. profiler: Done. Duration: 4ms , profilerQueryActiveList: Done. Duration: 1ms, , profilerSyncClusterInfo: Done. Duration: 1ms, profilerInvalidatedNodeList: Done. Duration: 0ms, profilerRemovedList: Done. Duration: 0ms,, profilerNewList: Done. Duration: 0ms, profilerInactiveList: Done. Duration: 1ms 2025-12-05 01:28:44,999 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-40:[ctx-40ff8694, job-7752/job-7753]) (logid:b7128a2c) Unable to complete AsyncJob {"accountId":2,"cmd":"com.cloud.vm.VmWorkStart","cmdInfo":"rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAJjXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAAJzcQB-AAgAAAAAAAAACXBwcHNxAH4ACAAAAAAAAAABcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAGnJPMEFCWFFBREhKVFluSlFUelZIVXpOS1JneHA","cmdVersion":0,"completeMsid":null,"created":"Fri Dec 05 01:28:38 EST 2025","id":7753,"initMsid":257839949501154,"instanceId":null,"instanceType":null,"lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"9a1350aa-a4e6-4c59-af08-11aea5007332"}, job origin: 7752 com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {"id":2445,"instanceName":"i-2-2445-VM","state":"Starting","type":"User","uuid":"6e37392e-8e4b-4167-b235-2c8e2cff6f33"}Scope=interface com.cloud.dc.DataCenter; id=1 at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1454) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5951) at jdk.internal.reflect.GeneratedMethodAccessor836.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:102) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:6075) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:698) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:646) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840) 2025-12-05 01:28:45,003 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-40:[ctx-40ff8694, job-7752/job-7753]) (logid:b7128a2c) Complete async job-7753, jobStatus: FAILED, resultCode: 0, result: rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uVk1PcAAAABACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvblZNT3AAAAAVAgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdAClVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNIGluc3RhbmNlIHsiaWQiOjI0NDUsImluc3RhbmNlTmFtZSI6ImktMi0yNDQ1LVZNIiwic3RhdGUiOiJTdGFydGluZyIsInR5cGUiOiJVc2VyIiwidXVpZCI6IjZlMzczOTJlLThlNGItNDE2Ny1iMjM1LTJjOGUyY2ZmNmYzMyJ9dXIAHltMamF2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50OwJGKjw8_SI5AgAAeHAAAAAUc3IAG2phdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudGEJxZomNt2FAgAIQgAGZm9ybWF0SQAKbGluZU51bWJlckwAD2NsYXNzTG9hZGVyTmFtZXEAfgAKTAAOZGVjbGFyaW5nQ2xhc3NxAH4ACkwACGZpbGVOYW1lcQB-AApMAAptZXRob2ROYW1lcQB-AApMAAptb2R1bGVOYW1lcQB-AApMAA1tb2R1bGVWZXJzaW9ucQB-AAp4cAEAAAWudAADYXBwdAAmY29tLmNsb3VkLnZtLlZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGx0AB5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsLmphdmF0ABBvcmNoZXN0cmF0ZVN0YXJ0cHBzcQB-ABEBAAAXP3EAfgATcQB-ABRxAH4AFXEAfgAWcHBzcQB-ABEA_____3B0AC9qZGsuaW50ZXJuYWwucmVmbGVjdC5HZW5lcmF0ZWRNZXRob2RBY2Nlc3NvcjgzNnB0AAZpbnZva2VwcHNxAH4AEQIAAAArcHQAMWpkay5pbnRlcm5hbC5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGnQACWphdmEuYmFzZXQABzE3LjAuMTdzcQB-ABECAAACOXB0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAacQB-AB5xAH4AH3NxAH4AEQEAAABmcQB-ABN0ACJjb20uY2xvdWQudm0uVm1Xb3JrSm9iSGFuZGxlclByb3h5dAAaVm1Xb3JrSm9iSGFuZGxlclByb3h5LmphdmF0AA9oYW5kbGVWbVdvcmtKb2JwcHNxAH4AEQEAABe7cQB-ABNxAH4AFHEAfgAVcQB-ACZwcHNxAH4AEQEAAABjcQB-ABN0ACBjb20uY2xvdWQudm0uVm1Xb3JrSm9iRGlzcGF0Y2hlcnQAGFZtV29ya0pvYkRpc3BhdGNoZXIuamF2YXQABnJ1bkpvYnBwc3EAfgARAQAAArpxAH4AE3QAP29yZy5hcGFjaGUuY2xvdWRzdGFjay5mcmFtZXdvcmsuam9icy5pbXBsLkFzeW5jSm9iTWFuYWdlckltcGwkNXQAGEFzeW5jSm9iTWFuYWdlckltcGwuamF2YXQADHJ1bkluQ29udGV4dHBwc3EAfgARAQAAADFxAH4AE3QAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVucHBzcQB-ABEBAAAAOHEAfgATdABCb3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5pbXBsLkRlZmF1bHRNYW5hZ2VkQ29udGV4dCQxdAAaRGVmYXVsdE1hbmFnZWRDb250ZXh0LmphdmF0AARjYWxscHBzcQB-ABEBAAAAZ3EAfgATdABAb3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5pbXBsLkRlZmF1bHRNYW5hZ2VkQ29udGV4dHEAfgA2dAAPY2FsbFdpdGhDb250ZXh0cHBzcQB-ABEBAAAANXEAfgATcQB-ADlxAH4ANnQADnJ1bldpdGhDb250ZXh0cHBzcQB-ABEBAAAALnEAfgATdAA8b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlcQB-ADJxAH4AM3Bwc3EAfgARAQAAAoZxAH4AE3EAfgAtcQB-AC5xAH4AM3Bwc3EAfgARAgAAAhtwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuRXhlY3V0b3JzJFJ1bm5hYmxlQWRhcHRlcnQADkV4ZWN1dG9ycy5qYXZhcQB-ADdxAH4AHnEAfgAfc3EAfgARAgAAAQhwdAAfamF2YS51dGlsLmNvbmN1cnJlbnQuRnV0dXJlVGFza3QAD0Z1dHVyZVRhc2suamF2YXEAfgAzcQB-AB5xAH4AH3NxAH4AEQIAAARwcHQAJ2phdmEudXRpbC5jb25jdXJyZW50LlRocmVhZFBvb2xFeGVjdXRvcnQAF1RocmVhZFBvb2xFeGVjdXRvci5qYXZhdAAJcnVuV29ya2VycQB-AB5xAH4AH3NxAH4AEQIAAAJ7cHQALmphdmEudXRpbC5jb25jdXJyZW50LlRocmVhZFBvb2xFeGVjdXRvciRXb3JrZXJxAH4ASHEAfgAzcQB-AB5xAH4AH3NxAH4AEQIAAANIcHQAEGphdmEubGFuZy5UaHJlYWR0AAtUaHJlYWQuamF2YXEAfgAzcQB-AB5xAH4AH3NyAB9qYXZhLnV0aWwuQ29sbGVjdGlvbnMkRW1wdHlMaXN0ergXtDynnt4CAAB4cHhzcgARamF2YS5sYW5nLkludGVnZXIS4qCk94GHOAIAAUkABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAQ73NyABNqYXZhLnV0aWwuQXJyYXlMaXN0eIHSHZnHYZ0DAAFJAARzaXpleHAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFIAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlchBQVG0DRR4AAgAAeHAA 2025-12-05 01:28:45,005 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-40:[ctx-40ff8694, job-7752/job-7753]) (logid:b7128a2c) Publish async job-7753 complete on message bus 2025-12-05 01:28:45,005 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-40:[ctx-40ff8694, job-7752/job-7753]) (logid:b7128a2c) Wake up jobs related to job-7753 2025-12-05 01:28:45,005 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-40:[ctx-40ff8694, job-7752/job-7753]) (logid:b7128a2c) Update db status for job-7753 2025-12-05 01:28:45,008 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-40:[ctx-40ff8694, job-7752/job-7753]) (logid:b7128a2c) Wake up jobs joined with job-7753 and disjoin all subjobs created from job- 7753 2025-12-05 01:28:45,038 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (Work-Job-Executor-40:[ctx-40ff8694, job-7752/job-7753]) (logid:b7128a2c) Done executing com.cloud.vm.VmWorkStart for job-7753 2025-12-05 01:28:45,041 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-40:[ctx-40ff8694, job-7752/job-7753]) (logid:b7128a2c) Remove job-7753 from job monitoring 2025-12-05 01:28:45,059 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752, ctx-e0421bd0]) (logid:b7128a2c) Destroying vm VM instance {"id":2445,"instanceName":"i-2-2445-VM","state":"Stopped","type":"User","uuid":"6e37392e-8e4b-4167-b235-2c8e2cff6f33"} as it failed to create on Host: null with id null 2025-12-05 01:28:45,079 DEBUG [c.c.a.ApiServlet] (qtp1404565079-16806:[ctx-2e0b2834]) (logid:db853320) ===START=== 10.20.0.103 -- GET listall=true&details=group,nics,secgrp,tmpl,servoff,diskoff,iso,volume,affgrp,backoff&isvnf=false&page=1&pagesize=20&showIcon=true&command=listVirtualMachines&response=json& 2025-12-05 01:28:45,079 DEBUG [c.c.a.ApiServlet] (qtp1404565079-16806:[ctx-2e0b2834]) (logid:db853320) Two factor authentication is already verified for the user 2, so skipping 2025-12-05 01:28:45,088 DEBUG [c.c.a.ApiServer] (qtp1404565079-16806:[ctx-2e0b2834, ctx-e6e4df1b]) (logid:db853320) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"2003d6d6-921a-11f0-8387-aeb808e635e9"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-12-05 01:28:45,093 INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1404565079-16806:[ctx-2e0b2834, ctx-e6e4df1b]) (logid:db853320) Account for user id 20046fa3-921a-11f0-8387-aeb808e635e9 is Root Admin or Domain Admin, all APIs are allowed. 2025-12-05 01:28:45,093 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1404565079-16806:[ctx-2e0b2834, ctx-e6e4df1b]) (logid:db853320) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-12-05 01:28:45,093 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1404565079-16806:[ctx-2e0b2834, ctx-e6e4df1b]) (logid:db853320) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-12-05 01:28:45,094 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752, ctx-e0421bd0]) (logid:b7128a2c) VM instance {"id":2445,"instanceName":"i-2-2445-VM","state":"Error","type":"User","uuid":"6e37392e-8e4b-4167-b235-2c8e2cff6f33"} state transited from [Stopped] to [Error] with event [OperationFailedToError]. VM's original host: null, new host: null, host before state transition: null 2025-12-05 01:28:45,108 DEBUG [o.a.c.e.o.VolumeOrchestrator] (API-Job-Executor-28:[ctx-188f0918, job-7752, ctx-e0421bd0, ctx-fead1c03]) (logid:b7128a2c) Found [0] snapshots [[]] that have checkpoints for volume with id [2469]. 2025-12-05 01:28:45,222 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752, ctx-e0421bd0, ctx-fead1c03]) (logid:b7128a2c) Updating resource Type = volume count for Account with id = 2 Operation = decreasing Amount = 1 2025-12-05 01:28:45,230 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752, ctx-e0421bd0, ctx-fead1c03]) (logid:b7128a2c) Updating resource Type = primary_storage count for Account with id = 2 Operation = decreasing Amount = (60.00 GB) 64424509440 2025-12-05 01:28:45,289 DEBUG [c.c.a.ApiServlet] (qtp1404565079-16806:[ctx-2e0b2834, ctx-e6e4df1b]) (logid:db853320) ===END=== 10.20.0.103 -- GET listall=true&details=group,nics,secgrp,tmpl,servoff,diskoff,iso,volume,affgrp,backoff&isvnf=false&page=1&pagesize=20&showIcon=true&command=listVirtualMachines&response=json& 2025-12-05 01:28:45,327 DEBUG [c.c.a.ApiServlet] (qtp1404565079-16769:[ctx-7aaaf5b8]) (logid:4b64f101) ===START=== 10.20.0.103 -- GET managementserverid=5104cb13-ee52-4ac0-979d-d9d7cb33ce44&command=readyForShutdown&response=json& 2025-12-05 01:28:45,327 DEBUG [c.c.a.ApiServlet] (qtp1404565079-16769:[ctx-7aaaf5b8]) (logid:4b64f101) Two factor authentication is already verified for the user 2, so skipping 2025-12-05 01:28:45,337 DEBUG [c.c.a.ApiServer] (qtp1404565079-16769:[ctx-7aaaf5b8, ctx-327a7bef]) (logid:4b64f101) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"2003d6d6-921a-11f0-8387-aeb808e635e9"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-12-05 01:28:45,341 INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1404565079-16769:[ctx-7aaaf5b8, ctx-327a7bef]) (logid:4b64f101) Account for user id 20046fa3-921a-11f0-8387-aeb808e635e9 is Root Admin or Domain Admin, all APIs are allowed. 2025-12-05 01:28:45,342 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1404565079-16769:[ctx-7aaaf5b8, ctx-327a7bef]) (logid:4b64f101) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-12-05 01:28:45,342 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1404565079-16769:[ctx-7aaaf5b8, ctx-327a7bef]) (logid:4b64f101) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-12-05 01:28:45,350 DEBUG [c.c.a.ApiServlet] (qtp1404565079-16769:[ctx-7aaaf5b8, ctx-327a7bef]) (logid:4b64f101) ===END=== 10.20.0.103 -- GET managementserverid=5104cb13-ee52-4ac0-979d-d9d7cb33ce44&command=readyForShutdown&response=json& 2025-12-05 01:28:45,372 WARN [c.c.a.AlertManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752, ctx-e0421bd0]) (logid:b7128a2c) alertType=[8] dataCenterId=[1] podId=[1] clusterId=[null] message=[Failed to deploy Vm VM instance {"id":2445,"instanceName":"i-2-2445-VM","state":"Error","type":"User","uuid":"6e37392e-8e4b-4167-b235-2c8e2cff6f33"}, on Host null with Id: null]. 2025-12-05 01:28:45,405 WARN [c.c.a.AlertManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752, ctx-e0421bd0]) (logid:b7128a2c) No recipients set in global setting 'alert.email.addresses', skipping sending alert with subject [Failed to deploy Vm VM instance {"id":2445,"instanceName":"i-2-2445-VM","state":"Error","type":"User","uuid":"6e37392e-8e4b-4167-b235-2c8e2cff6f33"}, on Host null with Id: null] and content [Failed to deploy Vm VM instance {"id":2445,"instanceName":"i-2-2445-VM","state":"Error","type":"User","uuid":"6e37392e-8e4b-4167-b235-2c8e2cff6f33"}, on Host null with Id: null]. 2025-12-05 01:28:45,409 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752, ctx-e0421bd0]) (logid:b7128a2c) Updating resource Type = user_vm count for Account with id = 2 Operation = decreasing Amount = 1 2025-12-05 01:28:45,418 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752, ctx-e0421bd0]) (logid:b7128a2c) Updating resource Type = cpu count for Account with id = 2 Operation = decreasing Amount = 2 2025-12-05 01:28:45,426 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752, ctx-e0421bd0]) (logid:b7128a2c) Updating resource Type = memory count for Account with id = 2 Operation = decreasing Amount = 4096 2025-12-05 01:28:45,563 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-28:[ctx-188f0918, job-7752]) (logid:b7128a2c) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM [6e37392e-8e4b-4167-b235-2c8e2cff6f33] due to [Unable to create a deployment for VM instance {"id":2445,"instanceName":"i-2-2445-VM","state":"Starting","type":"User","uuid":"6e37392e-8e4b-4167-b235-2c8e2cff6f33"}]. at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:935) at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:245) at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:214) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5822) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5674) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5275) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5264) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:109) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) at jdk.proxy3/jdk.proxy3.$Proxy219.startVirtualMachine(Unknown Source) at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:97) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:173) at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:110) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:698) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:646) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840) Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {"id":2445,"instanceName":"i-2-2445-VM","state":"Starting","type":"User","uuid":"6e37392e-8e4b-4167-b235-2c8e2cff6f33"}Scope=interface com.cloud.dc.DataCenter; id=1 at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1454) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5951) at jdk.internal.reflect.GeneratedMethodAccessor836.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:102) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:6075) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99) ... 12 more 2025-12-05 01:28:45,564 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752]) (logid:b7128a2c) Complete async job-7752, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Unable to start a VM [6e37392e-8e4b-4167-b235-2c8e2cff6f33] due to [Unable to create a deployment for VM instance {"id":2445,"instanceName":"i-2-2445-VM","state":"Starting","type":"User","uuid":"6e37392e-8e4b-4167-b235-2c8e2cff6f33"}]."} 2025-12-05 01:28:45,566 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752]) (logid:b7128a2c) Publish async job-7752 complete on message bus 2025-12-05 01:28:45,566 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752]) (logid:b7128a2c) Wake up jobs related to job-7752 2025-12-05 01:28:45,566 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752]) (logid:b7128a2c) Update db status for job-7752 2025-12-05 01:28:45,568 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-28:[ctx-188f0918, job-7752]) (logid:b7128a2c) Wake up jobs joined with job-7752 and disjoin all subjobs created from job- 7752 2025-12-05 01:28:45,582 DEBUG [c.c.a.ApiServlet] (qtp1404565079-3996:[ctx-f1085902]) (logid:dd215870) ===START=== 10.20.0.103 -- GET listall=true&details=group,nics,secgrp,tmpl,servoff,diskoff,iso,volume,affgrp,backoff&isvnf=false&page=1&pagesize=20&showIcon=true&command=listVirtualMachines&response=json& 2025-12-05 01:28:45,582 DEBUG [c.c.a.ApiServlet] (qtp1404565079-3996:[ctx-f1085902]) (logid:dd215870) Two factor authentication is already verified for the user 2, so skipping 2025-12-05 01:28:45,586 DEBUG [c.c.a.ApiServer] (API-Job-Executor-28:[ctx-188f0918, job-7752]) (logid:b7128a2c) Retrieved cmdEventType from job info: VM.CREATE 2025-12-05 01:28:45,590 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (API-Job-Executor-28:[ctx-188f0918, job-7752]) (logid:b712 Thank you! On Fri, Dec 5, 2025 at 5:30 PM Pearl d'Silva <[email protected]> wrote: > Hi Sanjay, > > Could you please share some logs or specifics on what is the exact issue > you are facing? > > Thanks, > Pearl > ________________________________ > From: Sanjay Kumar <[email protected]> > Sent: December 5, 2025 6:44 AM > To: [email protected] <[email protected]>; > [email protected] <[email protected]> > Subject: We are facing issue while using multi-ARC in same zone. > > Hello! > > We are using amd and arm with different cluster in same zone but facing > issue while deploying the vm in arm cluster. is there any missed or > something need to address or follow? > > ACS: 4.22 > KVM: ubuntu 22 > > Any help would be really appreciated. > > Thank you >
