Chandan Purushothama created CLOUDSTACK-7948:
------------------------------------------------
Summary: [Automation] Two "VOLUME.DELETE" Events are being
registered instead of one - On Destroying a User VM belonging to a Project
Key: CLOUDSTACK-7948
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7948
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Automation, Test
Affects Versions: 4.5.0
Reporter: Chandan Purushothama
Fix For: 4.5.0
==========================================
User VM in Project Events Information in the Database:
==========================================
{noformat}
mysql> select * from usage_event where account_id=6;
+----+-------------------------+------------+---------------------+---------+-------------+----------------+-------------+-------------+-------------+----------------+-----------+--------------+
| id | type | account_id | created | zone_id |
resource_id | resource_name | offering_id | template_id | size |
resource_type | processed | virtual_size |
+----+-------------------------+------------+---------------------+---------+-------------+----------------+-------------+-------------+-------------+----------------+-----------+--------------+
| 19 | TEMPLATE.CREATE | 6 | 2014-11-19 21:28:14 | 1 |
202 | TestTemplate-1 | NULL | NULL | 1708331520 | NULL
| 0 | 8589934592 |
| 20 | VOLUME.CREATE | 6 | 2014-11-19 21:32:23 | 1 |
9 | ROOT-9 | 1 | 5 | 21474836480 | NULL
| 0 | NULL |
| 21 | VM.CREATE | 6 | 2014-11-19 21:32:23 | 1 |
9 | Project-VM-1 | 1 | 5 | NULL |
XenServer | 0 | NULL |
| 22 | NET.IPASSIGN | 6 | 2014-11-19 21:32:25 | 1 |
6 | 10.219.196.151 | NULL | 0 | 0 |
DirectAttached | 0 | NULL |
| 23 | NETWORK.OFFERING.ASSIGN | 6 | 2014-11-19 21:32:33 | 1 |
9 | 19 | 6 | NULL | 1 | NULL
| 0 | NULL |
| 24 | SG.ASSIGN | 6 | 2014-11-19 21:32:33 | 1 |
9 | NULL | 4 | NULL | NULL | NULL
| 0 | NULL |
| 25 | VM.START | 6 | 2014-11-19 21:32:33 | 1 |
9 | Project-VM-1 | 1 | 5 | NULL |
XenServer | 0 | NULL |
| 26 | SG.REMOVE | 6 | 2014-11-19 21:33:10 | 1 |
9 | NULL | 4 | NULL | NULL | NULL
| 0 | NULL |
| 27 | VM.STOP | 6 | 2014-11-19 21:33:10 | 1 |
9 | Project-VM-1 | 1 | 5 | NULL |
XenServer | 0 | NULL |
| 28 | NETWORK.OFFERING.REMOVE | 6 | 2014-11-19 21:33:10 | 1 |
9 | 19 | 6 | NULL | 0 | NULL
| 0 | NULL |
| 31 | VM.DESTROY | 6 | 2014-11-20 00:08:37 | 1 |
9 | Project-VM-1 | 1 | 5 | NULL |
XenServer | 0 | NULL |
| 32 | VOLUME.DELETE | 6 | 2014-11-20 00:08:37 | 1 |
9 | ROOT-9 | NULL | NULL | NULL | NULL
| 0 | NULL |
| 33 | NET.IPRELEASE | 6 | 2014-11-20 00:08:39 | 1 |
6 | 10.219.196.151 | NULL | 0 | 0 |
DirectAttached | 0 | NULL |
| 34 | VOLUME.DELETE | 6 | 2014-11-20 00:08:39 | 1 |
9 | ROOT-9 | NULL | NULL | NULL | NULL
| 0 | NULL |
+----+-------------------------+------------+---------------------+---------+-------------+----------------+-------------+-------------+-------------+----------------+-----------+--------------+
14 rows in set (0.00 sec)
{noformat}
=================
Destroy VM Job Log:
=================
{noformat}
2014-11-20 00:08:35,021 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(catalina-exec-11:ctx-fb3a3d80 ctx-67753d29) submit async job-50, details:
AsyncJobVO {id:50, userId: 2, accountId: 2, instanceType: VirtualMachine,
instanceId: 9, cmd:
org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin, cmdInfo:
{"sessionkey":"KxWOb6uxo53LBRidu+yVRm8qK04\u003d","cmdEventType":"VM.DESTROY","ctxUserId":"2","httpmethod":"GET","id":"ede32a62-67d7-487a-bb20-12c53d6e7c5a","response":"json","ctxDetails":"{\"com.cloud.vm.VirtualMachine\":\"ede32a62-67d7-487a-bb20-12c53d6e7c5a\"}","projectid":"fa39cf9d-6d28-4673-8698-6781e59dbf6b","_":"1416442109090","expunge":"true","ctxAccountId":"2","uuid":"ede32a62-67d7-487a-bb20-12c53d6e7c5a","ctxStartEventId":"107"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 11627713415566, completeMsid: null, lastUpdated: null,
lastPolled: null, created: null}
2014-11-20 00:08:35,022 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-13:ctx-70d3ae4d job-50) Add job-50 into job monitoring
2014-11-20 00:08:35,022 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50) Executing AsyncJobVO {id:50, userId:
2, accountId: 2, instanceType: VirtualMachine, instanceId: 9, cmd:
org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin, cmdInfo:
{"sessionkey":"KxWOb6uxo53LBRidu+yVRm8qK04\u003d","cmdEventType":"VM.DESTROY","ctxUserId":"2","httpmethod":"GET","id":"ede32a62-67d7-487a-bb20-12c53d6e7c5a","response":"json","ctxDetails":"{\"com.cloud.vm.VirtualMachine\":\"ede32a62-67d7-487a-bb20-12c53d6e7c5a\"}","projectid":"fa39cf9d-6d28-4673-8698-6781e59dbf6b","_":"1416442109090","expunge":"true","ctxAccountId":"2","uuid":"ede32a62-67d7-487a-bb20-12c53d6e7c5a","ctxStartEventId":"107"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 11627713415566, completeMsid: null, lastUpdated: null,
lastPolled: null, created: null}
2014-11-20 00:08:35,022 DEBUG [c.c.a.ApiServlet] (catalina-exec-11:ctx-fb3a3d80
ctx-67753d29) ===END=== 10.252.113.187 -- GET
command=destroyVirtualMachine&response=json&sessionkey=KxWOb6uxo53LBRidu%2ByVRm8qK04%3D&projectid=fa39cf9d-6d28-4673-8698-6781e59dbf6b&id=ede32a62-67d7-487a-bb20-12c53d6e7c5a&expunge=true&_=1416442109090
2014-11-20 00:08:35,032 WARN [c.c.a.d.ParamGenericValidationWorker]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Received unknown
parameters for command destroyVirtualMachine. Unknown parameters : projectid
2014-11-20 00:08:35,053 DEBUG [c.c.v.VirtualMachineManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Destroying vm
VM[User|i-6-9-CUSTOMERTEST]
2014-11-20 00:08:35,063 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Sync job-51 execution on
object VmWorkJobQueue.9
2014-11-20 00:08:35,602 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-649609cc) Execute sync-queue item: SyncQueueItemVO
{id:19, queueId: 17, contentType: AsyncJob, contentId: 51, lastProcessMsid:
null, lastprocessNumber: null, lastProcessTime: null, created: Thu Nov 20
00:08:35 UTC 2014}
2014-11-20 00:08:35,603 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-649609cc) Schedule queued job-51
2014-11-20 00:08:35,607 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51) Add job-51 into job monitoring
2014-11-20 00:08:35,608 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51) Executing AsyncJobVO {id:51,
userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd:
com.cloud.vm.VmWorkStop, cmdInfo:
rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAACXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 11627713415566, completeMsid: null, lastUpdated: null,
lastPolled: null, created: Thu Nov 20 00:08:35 UTC 2014}
2014-11-20 00:08:35,609 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51) Run VM work job:
com.cloud.vm.VmWorkStop for VM 9, job origin: 50
2014-11-20 00:08:35,610 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51 ctx-2a95f6ae) Execute VM work
job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":2,"accountId":2,"vmId":9,"handlerName":"VirtualMachineManagerImpl"}
2014-11-20 00:08:35,616 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51 ctx-2a95f6ae) VM is already
stopped: VM[User|i-6-9-CUSTOMERTEST]
2014-11-20 00:08:35,616 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51 ctx-2a95f6ae) Done executing
VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":2,"accountId":2,"vmId":9,"handlerName":"VirtualMachineManagerImpl"}
2014-11-20 00:08:35,616 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51 ctx-2a95f6ae) Complete async
job-51, jobStatus: SUCCEEDED, resultCode: 0, result: null
2014-11-20 00:08:35,618 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51 ctx-2a95f6ae) Publish async
job-51 complete on message bus
2014-11-20 00:08:35,618 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51 ctx-2a95f6ae) Wake up jobs
related to job-51
2014-11-20 00:08:35,618 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51 ctx-2a95f6ae) Update db status
for job-51
2014-11-20 00:08:35,619 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51 ctx-2a95f6ae) Wake up jobs
joined with job-51 and disjoin all subjobs created from job- 51
2014-11-20 00:08:35,624 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51) Done with run of VM work job:
com.cloud.vm.VmWorkStop for VM 9, job origin: 50
2014-11-20 00:08:35,624 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51) Done executing
com.cloud.vm.VmWorkStop for job-51
2014-11-20 00:08:35,627 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51) Sync queue (17) is currently
empty
2014-11-20 00:08:35,628 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-14:ctx-eb594f01 job-50/job-51) Remove job-51 from job
monitoring
2014-11-20 00:08:35,638 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Sync job-52 execution on
object VmWorkJobQueue.9
2014-11-20 00:08:35,659 DEBUG [c.c.s.StatsCollector]
(StatsCollector-2:ctx-4020448b) AutoScaling Monitor is running...
2014-11-20 00:08:37,601 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-908f32e6) Execute sync-queue item: SyncQueueItemVO
{id:20, queueId: 17, contentType: AsyncJob, contentId: 52, lastProcessMsid:
null, lastprocessNumber: null, lastProcessTime: null, created: Thu Nov 20
00:08:35 UTC 2014}
2014-11-20 00:08:37,603 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-908f32e6) Schedule queued job-52
2014-11-20 00:08:37,618 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52) Add job-52 into job monitoring
2014-11-20 00:08:37,618 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52) Executing AsyncJobVO {id:52,
userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd:
com.cloud.vm.snapshot.VmWorkDeleteAllVMSnapshots, cmdInfo:
rO0ABXNyADBjb20uY2xvdWQudm0uc25hcHNob3QuVm1Xb3JrRGVsZXRlQWxsVk1TbmFwc2hvdHOsl-VRajf8cAIAAUwABHR5cGV0ACdMY29tL2Nsb3VkL3ZtL3NuYXBzaG90L1ZNU25hcHNob3QkVHlwZTt4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1ldAASTGphdmEvbGFuZy9TdHJpbmc7eHAAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAl0ABVWTVNuYXBzaG90TWFuYWdlckltcGxw,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 11627713415566, completeMsid: null, lastUpdated: null,
lastPolled: null, created: Thu Nov 20 00:08:35 UTC 2014}
2014-11-20 00:08:37,618 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52) Run VM work job:
com.cloud.vm.snapshot.VmWorkDeleteAllVMSnapshots for VM 9, job origin: 50
2014-11-20 00:08:37,621 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52 ctx-8bca0dc3) Execute VM work
job:
com.cloud.vm.snapshot.VmWorkDeleteAllVMSnapshots{"userId":2,"accountId":2,"vmId":9,"handlerName":"VMSnapshotManagerImpl"}
2014-11-20 00:08:37,630 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-ce4d53c8) Begin cleanup expired async-jobs
2014-11-20 00:08:37,637 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-ce4d53c8) End cleanup expired async-jobs
2014-11-20 00:08:37,661 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52 ctx-8bca0dc3) Done executing
VM work job:
com.cloud.vm.snapshot.VmWorkDeleteAllVMSnapshots{"userId":2,"accountId":2,"vmId":9,"handlerName":"VMSnapshotManagerImpl"}
2014-11-20 00:08:37,661 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52 ctx-8bca0dc3) Complete async
job-52, jobStatus: SUCCEEDED, resultCode: 0, result:
rO0ABXNyABFqYXZhLmxhbmcuQm9vbGVhbs0gcoDVnPruAgABWgAFdmFsdWV4cAE
2014-11-20 00:08:37,663 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52 ctx-8bca0dc3) Publish async
job-52 complete on message bus
2014-11-20 00:08:37,663 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52 ctx-8bca0dc3) Wake up jobs
related to job-52
2014-11-20 00:08:37,663 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52 ctx-8bca0dc3) Update db status
for job-52
2014-11-20 00:08:37,664 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52 ctx-8bca0dc3) Wake up jobs
joined with job-52 and disjoin all subjobs created from job- 52
2014-11-20 00:08:37,679 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52) Done with run of VM work job:
com.cloud.vm.snapshot.VmWorkDeleteAllVMSnapshots for VM 9, job origin: 50
2014-11-20 00:08:37,680 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52) Done executing
com.cloud.vm.snapshot.VmWorkDeleteAllVMSnapshots for job-52
2014-11-20 00:08:37,683 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52) Sync queue (17) is currently
empty
2014-11-20 00:08:37,684 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-15:ctx-4d461fe6 job-50/job-52) Remove job-52 from job
monitoring
2014-11-20 00:08:37,702 DEBUG [c.c.c.CapacityManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) VM state transitted from
:Stopped to Destroyed with event: DestroyRequestedvm's original host id: 1 new
host id: null host id before state transition: null
2014-11-20 00:08:37,709 DEBUG [c.c.c.CapacityManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Hosts's actual total
CPU: 22400 and CPU after applying overprovisioning: 22400
2014-11-20 00:08:37,709 DEBUG [c.c.c.CapacityManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Hosts's actual total
RAM: 7434305920 and RAM after applying overprovisioning: 7434306048
2014-11-20 00:08:37,709 DEBUG [c.c.c.CapacityManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) release cpu from host:
1, old used: 2000,reserved: 0, actual total: 22400, total with
overprovisioning: 22400; new used: 2000,reserved:0; movedfromreserved:
true,moveToReserveredfalse
2014-11-20 00:08:37,709 DEBUG [c.c.c.CapacityManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) release mem from host:
1, old used: 2281701376,reserved: 0, total: 7434306048; new used:
2281701376,reserved:0; movedfromreserved: true,moveToReserveredfalse
2014-11-20 00:08:37,747 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Sync job-53 execution on
object VmWorkJobQueue.9
2014-11-20 00:08:37,864 DEBUG [c.c.s.StatsCollector]
(StatsCollector-4:ctx-f85ede2e) VmStatsCollector is running...
2014-11-20 00:08:37,878 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-378:ctx-72ebbb5a) Seq 1-6620572927211344033: Executing request
2014-11-20 00:08:37,901 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-378:ctx-72ebbb5a) Vm cpu utilization 0.10777777777777779
2014-11-20 00:08:37,901 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-378:ctx-72ebbb5a) Seq 1-6620572927211344033: Response Received:
2014-11-20 00:08:37,901 DEBUG [c.c.a.t.Request] (StatsCollector-4:ctx-f85ede2e)
Seq 1-6620572927211344033: Received: { Ans: , MgmtId: 11627713415566, via: 1,
Ver: v1, Flags: 10, { GetVmStatsAnswer } }
2014-11-20 00:08:38,066 DEBUG [c.c.a.ApiServlet] (catalina-exec-5:ctx-15a13cc5)
===START=== 10.252.113.187 -- GET
command=queryAsyncJobResult&jobId=d0617b23-fc22-44ba-8d97-8e9b5c2889fb&response=json&sessionkey=KxWOb6uxo53LBRidu%2ByVRm8qK04%3D&projectid=fa39cf9d-6d28-4673-8698-6781e59dbf6b&_=1416442112364
2014-11-20 00:08:38,074 WARN [c.c.a.d.ParamGenericValidationWorker]
(catalina-exec-5:ctx-15a13cc5 ctx-a8738fbc) Received unknown parameters for
command queryAsyncJobResult. Unknown parameters : projectid
2014-11-20 00:08:38,086 DEBUG [c.c.a.ApiServlet] (catalina-exec-5:ctx-15a13cc5
ctx-a8738fbc) ===END=== 10.252.113.187 -- GET
command=queryAsyncJobResult&jobId=d0617b23-fc22-44ba-8d97-8e9b5c2889fb&response=json&sessionkey=KxWOb6uxo53LBRidu%2ByVRm8qK04%3D&projectid=fa39cf9d-6d28-4673-8698-6781e59dbf6b&_=1416442112364
2014-11-20 00:08:38,845 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-4:null) SeqA 3-13515: Processing Seq 3-13515: { Cmd ,
MgmtId: -1, via: 3, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2014-11-20 00:08:38,849 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-4:null) SeqA 3-13515: Sending Seq 3-13515: { Ans: ,
MgmtId: 11627713415566, via: 3, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2014-11-20 00:08:39,602 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-d2cae0fb) Execute sync-queue item: SyncQueueItemVO
{id:21, queueId: 17, contentType: AsyncJob, contentId: 53, lastProcessMsid:
null, lastprocessNumber: null, lastProcessTime: null, created: Thu Nov 20
00:08:37 UTC 2014}
2014-11-20 00:08:39,604 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-d2cae0fb) Schedule queued job-53
2014-11-20 00:08:39,608 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53) Add job-53 into job monitoring
2014-11-20 00:08:39,608 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53) Executing AsyncJobVO {id:53,
userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd:
com.cloud.vm.VmWorkStop, cmdInfo:
rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAACXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 11627713415566, completeMsid: null, lastUpdated: null,
lastPolled: null, created: Thu Nov 20 00:08:37 UTC 2014}
2014-11-20 00:08:39,610 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53) Run VM work job:
com.cloud.vm.VmWorkStop for VM 9, job origin: 50
2014-11-20 00:08:39,611 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53 ctx-c1bf9be1) Execute VM work
job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":2,"accountId":2,"vmId":9,"handlerName":"VirtualMachineManagerImpl"}
2014-11-20 00:08:39,615 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53 ctx-c1bf9be1) Stopped called
on VM[User|i-6-9-CUSTOMERTEST] but the state is Destroyed
2014-11-20 00:08:39,615 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53 ctx-c1bf9be1) Done executing
VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":2,"accountId":2,"vmId":9,"handlerName":"VirtualMachineManagerImpl"}
2014-11-20 00:08:39,615 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53 ctx-c1bf9be1) Complete async
job-53, jobStatus: SUCCEEDED, resultCode: 0, result: null
2014-11-20 00:08:39,617 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53 ctx-c1bf9be1) Publish async
job-53 complete on message bus
2014-11-20 00:08:39,617 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53 ctx-c1bf9be1) Wake up jobs
related to job-53
2014-11-20 00:08:39,617 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53 ctx-c1bf9be1) Update db status
for job-53
2014-11-20 00:08:39,618 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53 ctx-c1bf9be1) Wake up jobs
joined with job-53 and disjoin all subjobs created from job- 53
2014-11-20 00:08:39,628 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53) Done with run of VM work job:
com.cloud.vm.VmWorkStop for VM 9, job origin: 50
2014-11-20 00:08:39,628 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53) Done executing
com.cloud.vm.VmWorkStop for job-53
2014-11-20 00:08:39,639 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53) Sync queue (17) is currently
empty
2014-11-20 00:08:39,641 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-16:ctx-dd0514f7 job-50/job-53) Remove job-53 from job
monitoring
2014-11-20 00:08:39,645 DEBUG [c.c.c.CapacityManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) VM state transitted from
:Destroyed to Expunging with event: ExpungeOperationvm's original host id: 1
new host id: null host id before state transition: null
2014-11-20 00:08:39,645 DEBUG [c.c.v.VirtualMachineManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Destroying vm
VM[User|i-6-9-CUSTOMERTEST]
2014-11-20 00:08:39,645 DEBUG [c.c.v.VirtualMachineManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Cleaning up NICS
2014-11-20 00:08:39,645 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Cleaning network for vm:
9
2014-11-20 00:08:39,659 DEBUG [c.c.n.g.DirectNetworkGuru]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Deallocate network:
networkId: 204, ip: 10.219.196.151
2014-11-20 00:08:39,699 DEBUG [c.c.n.g.DirectNetworkGuru]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) remove nic 19 secondary
ip
2014-11-20 00:08:39,704 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Removed nic id=19
2014-11-20 00:08:39,705 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Revoving nic secondary
ip entry ...
2014-11-20 00:08:39,706 DEBUG [c.c.v.VirtualMachineManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Cleaning up hypervisor
data structures (ex. SRs in XenServer) for managed storage
2014-11-20 00:08:39,715 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Cleaning storage for vm:
9
2014-11-20 00:08:39,754 DEBUG [c.c.a.t.Request]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Seq
1-6620572927211344034: Sending { Cmd , MgmtId: 11627713415566, via: 1(bear),
Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"165eb875-341d-4d75-be5e-9dcc2e10a779","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"fe44f477-1a11-317e-be90-8add5f52fbd9","id":1,"poolType":"NetworkFilesystem","host":"10.219.192.40","path":"/vol/exports/901529-XrA7qT","port":2049,"url":"NetworkFilesystem://10.219.192.40/vol/exports/901529-XrA7qT/?ROLE=Primary&STOREUUID=fe44f477-1a11-317e-be90-8add5f52fbd9"}},"name":"ROOT-9","size":21474836480,"path":"017150f6-f7fd-4e50-9bb6-d853b0577e61","volumeId":9,"vmName":"i-6-9-CUSTOMERTEST","accountId":6,"format":"VHD","provisioningType":"THIN","id":9,"deviceId":0,"hypervisorType":"XenServer"}},"wait":0}}]
}
2014-11-20 00:08:39,755 DEBUG [c.c.a.t.Request]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Seq
1-6620572927211344034: Executing: { Cmd , MgmtId: 11627713415566, via:
1(bear), Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"165eb875-341d-4d75-be5e-9dcc2e10a779","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"fe44f477-1a11-317e-be90-8add5f52fbd9","id":1,"poolType":"NetworkFilesystem","host":"10.219.192.40","path":"/vol/exports/901529-XrA7qT","port":2049,"url":"NetworkFilesystem://10.219.192.40/vol/exports/901529-XrA7qT/?ROLE=Primary&STOREUUID=fe44f477-1a11-317e-be90-8add5f52fbd9"}},"name":"ROOT-9","size":21474836480,"path":"017150f6-f7fd-4e50-9bb6-d853b0577e61","volumeId":9,"vmName":"i-6-9-CUSTOMERTEST","accountId":6,"format":"VHD","provisioningType":"THIN","id":9,"deviceId":0,"hypervisorType":"XenServer"}},"wait":0}}]
}
2014-11-20 00:08:39,755 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-95:ctx-1785c8ca) Seq 1-6620572927211344034: Executing request
2014-11-20 00:08:39,945 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-95:ctx-1785c8ca) Seq 1-6620572927211344034: Response Received:
2014-11-20 00:08:39,946 DEBUG [c.c.a.t.Request] (DirectAgent-95:ctx-1785c8ca)
Seq 1-6620572927211344034: Processing: { Ans: , MgmtId: 11627713415566, via:
1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
2014-11-20 00:08:39,946 DEBUG [c.c.a.t.Request]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Seq
1-6620572927211344034: Received: { Ans: , MgmtId: 11627713415566, via: 1, Ver:
v1, Flags: 10, { Answer } }
2014-11-20 00:08:39,955 INFO [o.a.c.s.v.VolumeServiceImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Volume 9 is not referred
anywhere, remove it from volumes table
2014-11-20 00:08:39,977 DEBUG [c.c.v.VirtualMachineManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Expunged
VM[User|i-6-9-CUSTOMERTEST]
2014-11-20 00:08:39,984 DEBUG [c.c.u.AccountManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Access granted to
Acct[dde8b620-6f53-11e4-9960-0a9349ba498e-admin] to Domain:1/ by
AffinityGroupAccessChecker
2014-11-20 00:08:39,993 DEBUG [c.c.v.UserVmManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Starting cleaning up vm
VM[User|i-6-9-CUSTOMERTEST] resources...
2014-11-20 00:08:40,013 INFO [c.c.n.s.SecurityGroupManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Disassociated 1 network
groups from uservm 9
2014-11-20 00:08:40,015 DEBUG [c.c.n.s.SecurityGroupManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Security group mappings
are removed successfully for vm id=9
2014-11-20 00:08:40,033 DEBUG [c.c.n.f.FirewallManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) No firewall rules are
found for vm id=9
2014-11-20 00:08:40,039 DEBUG [c.c.v.UserVmManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Firewall rules are
removed successfully as a part of vm id=9 expunge
2014-11-20 00:08:40,043 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) No port forwarding rules
are found for vm id=9
2014-11-20 00:08:40,043 DEBUG [c.c.v.UserVmManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Port forwarding rules
are removed successfully as a part of vm id=9 expunge
2014-11-20 00:08:40,063 DEBUG [c.c.v.UserVmManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Removed vm id=9 from all
load balancers as a part of expunge process
2014-11-20 00:08:40,065 DEBUG [c.c.v.UserVmManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Successfully cleaned up
vm VM[User|i-6-9-CUSTOMERTEST] resources as a part of expunge process
2014-11-20 00:08:40,084 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-13:ctx-70d3ae4d job-50 ctx-012a756f) Complete async job-50,
jobStatus: SUCCEEDED, resultCode: 0, result:
org.apache.cloudstack.api.response.UserVmResponse/null/{"securitygroup":[],"nic":[],"tags":[],"affinitygroup":[]}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
