Hossein4185 commented on issue #7644:
URL: https://github.com/apache/cloudstack/issues/7644#issuecomment-1596657620
**The following log is the full server error:**
2023-06-19 10:34:50,252 DEBUG [c.c.a.ApiServlet]
(qtp1709366259-11:ctx-601ff485) (logid:30a99e62) ===START=== 192.168.123.19 --
GET
command=queryAsyncJobResult&jobId=c02aa83d-dd78-4922-9a0e-8915c4a73d3b&response=json&_=1687158100241
2023-06-19 10:34:50,272 DEBUG [c.c.a.ApiServer]
(qtp1709366259-11:ctx-601ff485 ctx-d9ca0fd4) (logid:30a99e62) CIDRs from which
account 'Acct[a27ca787-edb8-4361-b690-cb2f4cd95663-ho.karami]' is allowed to
perform API calls: 0.0.0.0/0,::/0
2023-06-19 10:34:50,320 DEBUG [c.c.a.ApiServlet]
(qtp1709366259-11:ctx-601ff485 ctx-d9ca0fd4) (logid:30a99e62) ===END===
192.168.123.19 -- GET
command=queryAsyncJobResult&jobId=c02aa83d-dd78-4922-9a0e-8915c4a73d3b&response=json&_=1687158100241
2023-06-19 10:34:50,467 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-beb3c5d9) (logid:72557208) Execute sync-queue
item: SyncQueueItemVO {id:382, queueId: 7894, contentType: AsyncJob, contentId:
510661, lastProcessMsid: 345051619551, lastprocessNumber: 20, lastProcessTime:
Mon Jun 19 10:34:50 IRST 2023, created: Mon Jun 19 10:34:50 IRST 2023}
2023-06-19 10:34:50,471 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-beb3c5d9) (logid:72557208) Schedule queued
job-510661
2023-06-19 10:34:50,486 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661) (logid:062ad570) Add
job-510661 into job monitoring
2023-06-19 10:34:50,500 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661) (logid:c02aa83d)
Executing AsyncJobVO {id:510661, userId: 39427, accountId: 19870, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo:
rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAE2eAAAAAAAAmgMAAAAAAAABz3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 345051619551, completeMsid: null, lastUpdated: null,
lastPolled: null, created: Mon Jun 19 10:34:50 IRST 2023, removed: null}
2023-06-19 10:34:50,501 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661) (logid:c02aa83d) Run
VM work job: com.cloud.vm.VmWorkStop for VM 463, job origin: 510652
2023-06-19 10:34:50,510 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661 ctx-c2a1bb00)
(logid:c02aa83d) Execute VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":39427,"accountId":19870,"vmId":463,"handlerName":"VirtualMachineManagerImpl"}
2023-06-19 10:34:50,525 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661 ctx-c2a1bb00)
(logid:c02aa83d) Stopped called on VM[User|i-943-463-VM] but the state is
Expunging
2023-06-19 10:34:50,525 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661 ctx-c2a1bb00)
(logid:c02aa83d) Done executing VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":39427,"accountId":19870,"vmId":463,"handlerName":"VirtualMachineManagerImpl"}
2023-06-19 10:34:50,529 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661 ctx-c2a1bb00)
(logid:c02aa83d) Complete async job-510661, jobStatus: SUCCEEDED, resultCode:
0, result: null
2023-06-19 10:34:50,534 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661 ctx-c2a1bb00)
(logid:c02aa83d) Publish async job-510661 complete on message bus
2023-06-19 10:34:50,535 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661 ctx-c2a1bb00)
(logid:c02aa83d) Wake up jobs related to job-510661
2023-06-19 10:34:50,535 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661 ctx-c2a1bb00)
(logid:c02aa83d) Update db status for job-510661
2023-06-19 10:34:50,539 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661 ctx-c2a1bb00)
(logid:c02aa83d) Wake up jobs joined with job-510661 and disjoin all subjobs
created from job- 510661
2023-06-19 10:34:50,574 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661) (logid:c02aa83d) Done
with run of VM work job: com.cloud.vm.VmWorkStop for VM 463, job origin: 510652
2023-06-19 10:34:50,574 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661) (logid:c02aa83d) Done
executing com.cloud.vm.VmWorkStop for job-510661
2023-06-19 10:34:50,584 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-1:ctx-f056e924 job-510652/job-510661) (logid:c02aa83d)
Remove job-510661 from job monitoring
2023-06-19 10:34:50,659 DEBUG [c.c.c.CapacityManagerImpl]
(API-Job-Executor-6:ctx-822bb171 job-510652 ctx-546500c6) (logid:c02aa83d) VM
state transitted from :Expunging to Expunging with event: ExpungeOperationvm's
original host id: 193 new host id: null host id before state transition: null
2023-06-19 10:34:50,676 DEBUG [c.c.v.VirtualMachineManagerImpl]
(API-Job-Executor-6:ctx-822bb171 job-510652 ctx-546500c6) (logid:c02aa83d)
Destroying vm VM[User|i-943-463-VM]
2023-06-19 10:34:50,676 DEBUG [c.c.v.VirtualMachineManagerImpl]
(API-Job-Executor-6:ctx-822bb171 job-510652 ctx-546500c6) (logid:c02aa83d)
Cleaning up NICS
2023-06-19 10:34:50,716 ERROR [c.c.a.ApiAsyncJobDispatcher]
(API-Job-Executor-6:ctx-822bb171 job-510652) (logid:c02aa83d) Unexpected
exception while executing
org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin
java.lang.NullPointerException
at
com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:528)
at
com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:494)
at
com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:2203)
at
com.cloud.vm.UserVmManagerImpl.destroyVm(UserVmManagerImpl.java:2888)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174)
at
com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy171.destroyVm(Unknown Source)
at
org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin.execute(DestroyVMCmdByAdmin.java:46)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:603)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:551)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-06-19 10:34:50,728 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-6:ctx-822bb171 job-510652) (logid:c02aa83d) Complete async
job-510652, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530}
2023-06-19 10:34:50,732 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-6:ctx-822bb171 job-510652) (logid:c02aa83d) Publish async
job-510652 complete on message bus
2023-06-19 10:34:50,732 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-6:ctx-822bb171 job-510652) (logid:c02aa83d) Wake up jobs
related to job-510652
2023-06-19 10:34:50,732 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-6:ctx-822bb171 job-510652) (logid:c02aa83d) Update db status
for job-510652
2023-06-19 10:34:50,736 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-6:ctx-822bb171 job-510652) (logid:c02aa83d) Wake up jobs
joined with job-510652 and disjoin all subjobs created from job- 510652
2023-06-19 10:34:50,748 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-6:ctx-822bb171 job-510652) (logid:c02aa83d) Done executing
org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin for job-510652
2023-06-19 10:34:50,748 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-6:ctx-822bb171 job-510652) (logid:c02aa83d) Remove job-510652
from job monitoring
2023-06-19 10:34:52,439 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-cb3f5c89) (logid:2d3418e7) Begin cleanup expired
async-jobs
2023-06-19 10:34:52,453 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-cb3f5c89) (logid:2d3418e7) End cleanup expired
async-jobs
2023-06-19 10:34:53,263 DEBUG [c.c.a.ApiServlet]
(qtp1709366259-15:ctx-506fac14) (logid:2158719e) ===START=== 192.168.123.19 --
GET
command=queryAsyncJobResult&jobId=c02aa83d-dd78-4922-9a0e-8915c4a73d3b&response=json&_=1687158100242
2023-06-19 10:34:53,284 DEBUG [c.c.a.ApiServer]
(qtp1709366259-15:ctx-506fac14 ctx-b71b52f6) (logid:2158719e) CIDRs from which
account 'Acct[a27ca787-edb8-4361-b690-cb2f4cd95663-ho.karami]' is allowed to
perform API calls: 0.0.0.0/0,::/0
2023-06-19 10:34:53,318 DEBUG [c.c.a.ApiServlet]
(qtp1709366259-15:ctx-506fac14 ctx-b71b52f6) (logid:2158719e) ===END===
192.168.123.19 -- GET
command=queryAsyncJobResult&jobId=c02aa83d-dd78-4922-9a0e-8915c4a73d3b&response=json&_=1687158100242
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
