Hello All,
I have Cloud stack version 4.8.0 with basic zone xenserver 6.5 , I have 3
secondary storages two of them working fine but I have NFS shared from
xenserver as secondary storage .
After restart the xenserver that have the secondary storage a problem occur
when create any instance from templates that exist in secondary storage inside
the xenserver , I attached log for the job , I believe the error is as below
"2016-10-04 10:16:36,145 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:6] is
unreachable: Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]"
please advice how can I fix this problem in order to be able to create any new
instance from the templates inside this secondary storage
Best Regards,
Ghaith Bannoura
2016-10-04 10:15:24,747 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking SecurityGroupProvider to prepare for
Nic[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181]
2016-10-04 10:15:24,751 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking VirtualRouter to prepare for
Nic[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181]
2016-10-04 10:15:34,679 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Checking if we need to prepare 1 volumes for
VM[User|i-14-2619-VM]
2016-10-04 10:15:36,029 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:4] is
unreachable: Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
2016-10-04 10:15:36,037 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Cleaning up resources for the vm VM[User|i-14-2619-VM] in
Starting state
2016-10-04 10:15:36,041 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Seq 8-8502233146522083298: Sending { Cmd , MgmtId:
144344642087, via: 8(xenserver10), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-14-2619-VM","wait":0}}]
}
2016-10-04 10:15:36,041 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Seq 8-8502233146522083298: Executing: { Cmd , MgmtId:
144344642087, via: 8(xenserver10), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-14-2619-VM","wait":0}}]
}
2016-10-04 10:15:36,067 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking SecurityGroupProvider to release
NicProfile[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181-vlan://untagged
2016-10-04 10:15:36,067 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking VirtualRouter to release
NicProfile[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181-vlan://untagged
2016-10-04 10:15:36,067 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Successfully released network resources for the vm
VM[User|i-14-2619-VM]
2016-10-04 10:15:36,067 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Successfully cleanued up resources for the vm
VM[User|i-14-2619-VM] in Starting state
2016-10-04 10:15:36,142 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Checking pool: 7 for volume allocation
[Vol[3392|vm=2619|ROOT]], maxSize : 1170497613824, totalAllocatedSize :
910627196928, askingSize : 53687091200, allocated disable threshold: 0.9
2016-10-04 10:15:36,144 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Deployment found - P0=VM[User|i-14-2619-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(6)-Host(11)-Storage(Volume(3392|ROOT-->Pool(7))]
2016-10-04 10:15:36,184 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking SecurityGroupProvider to prepare for
Nic[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181]
2016-10-04 10:15:36,188 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking VirtualRouter to prepare for
Nic[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181]
2016-10-04 10:15:49,796 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Checking if we need to prepare 1 volumes for
VM[User|i-14-2619-VM]
2016-10-04 10:15:51,040 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:7] is
unreachable: Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
2016-10-04 10:15:51,042 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Cleaning up resources for the vm VM[User|i-14-2619-VM] in
Starting state
2016-10-04 10:15:51,045 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Seq 11-683984193406902181: Sending { Cmd , MgmtId:
144344642087, via: 11(xenserver1), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-14-2619-VM","wait":0}}]
}
2016-10-04 10:15:51,045 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Seq 11-683984193406902181: Executing: { Cmd , MgmtId:
144344642087, via: 11(xenserver1), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-14-2619-VM","wait":0}}]
}
2016-10-04 10:15:51,069 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking SecurityGroupProvider to release
NicProfile[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181-vlan://untagged
2016-10-04 10:15:51,069 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking VirtualRouter to release
NicProfile[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181-vlan://untagged
2016-10-04 10:15:51,069 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Successfully released network resources for the vm
VM[User|i-14-2619-VM]
2016-10-04 10:15:51,069 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Successfully cleanued up resources for the vm
VM[User|i-14-2619-VM] in Starting state
2016-10-04 10:15:51,143 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Checking pool: 3 for volume allocation
[Vol[3392|vm=2619|ROOT]], maxSize : 3588696834048, totalAllocatedSize :
2455228121088, askingSize : 53687091200, allocated disable threshold: 0.9
2016-10-04 10:15:51,162 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Deployment found - P0=VM[User|i-14-2619-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(2)-Host(7)-Storage(Volume(3392|ROOT-->Pool(3))]
2016-10-04 10:15:51,205 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking SecurityGroupProvider to prepare for
Nic[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181]
2016-10-04 10:15:51,210 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking VirtualRouter to prepare for
Nic[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181]
2016-10-04 10:15:58,446 WARN [c.c.v.VirtualMachinePowerStateSyncImpl]
(DirectAgentCronJob-314:ctx-842e92e6) (logid:555126ac) VM state was updated but
update time is null?! vm id: 2619
2016-10-04 10:15:58,446 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(DirectAgentCronJob-314:ctx-842e92e6) (logid:555126ac) Detected missing VM.
host: 7, vm id: 2619, power state: PowerReportMissing, last state update:
1475565358444
2016-10-04 10:15:58,446 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(DirectAgentCronJob-314:ctx-842e92e6) (logid:555126ac) vm id: 2619 - time since
last state update(0ms) has not passed graceful period yet
2016-10-04 10:16:01,022 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Checking if we need to prepare 1 volumes for
VM[User|i-14-2619-VM]
2016-10-04 10:16:02,253 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:3] is
unreachable: Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
2016-10-04 10:16:02,256 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Cleaning up resources for the vm VM[User|i-14-2619-VM] in
Starting state
2016-10-04 10:16:02,258 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Seq 7-291045125918826486: Sending { Cmd , MgmtId:
144344642087, via: 7(xenserver9), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-14-2619-VM","wait":0}}]
}
2016-10-04 10:16:02,258 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Seq 7-291045125918826486: Executing: { Cmd , MgmtId:
144344642087, via: 7(xenserver9), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-14-2619-VM","wait":0}}]
}
2016-10-04 10:16:02,282 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking SecurityGroupProvider to release
NicProfile[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181-vlan://untagged
2016-10-04 10:16:02,283 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking VirtualRouter to release
NicProfile[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181-vlan://untagged
2016-10-04 10:16:02,283 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Successfully released network resources for the vm
VM[User|i-14-2619-VM]
2016-10-04 10:16:02,283 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Successfully cleanued up resources for the vm
VM[User|i-14-2619-VM] in Starting state
2016-10-04 10:16:02,348 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Checking pool: 1 for volume allocation
[Vol[3392|vm=2619|ROOT]], maxSize : 5388560760832, totalAllocatedSize :
3744951435264, askingSize : 53687091200, allocated disable threshold: 0.9
2016-10-04 10:16:02,351 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Deployment found - P0=VM[User|i-14-2619-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(3)-Storage(Volume(3392|ROOT-->Pool(1))]
2016-10-04 10:16:02,399 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking SecurityGroupProvider to prepare for
Nic[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181]
2016-10-04 10:16:02,403 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking VirtualRouter to prepare for
Nic[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181]
2016-10-04 10:16:12,211 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Checking if we need to prepare 1 volumes for
VM[User|i-14-2619-VM]
2016-10-04 10:16:13,653 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is
unreachable: Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
2016-10-04 10:16:13,655 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Cleaning up resources for the vm VM[User|i-14-2619-VM] in
Starting state
2016-10-04 10:16:13,658 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Seq 3-6681090047204139014: Sending { Cmd , MgmtId:
144344642087, via: 3(xenserver7), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-14-2619-VM","wait":0}}]
}
2016-10-04 10:16:13,658 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Seq 3-6681090047204139014: Executing: { Cmd , MgmtId:
144344642087, via: 3(xenserver7), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-14-2619-VM","wait":0}}]
}
2016-10-04 10:16:13,685 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking SecurityGroupProvider to release
NicProfile[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181-vlan://untagged
2016-10-04 10:16:13,685 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking VirtualRouter to release
NicProfile[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181-vlan://untagged
2016-10-04 10:16:13,685 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Successfully released network resources for the vm
VM[User|i-14-2619-VM]
2016-10-04 10:16:13,686 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Successfully cleanued up resources for the vm
VM[User|i-14-2619-VM] in Starting state
2016-10-04 10:16:13,748 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Checking pool: 5 for volume allocation
[Vol[3392|vm=2619|ROOT]], maxSize : 3588696834048, totalAllocatedSize :
2493421453312, askingSize : 53687091200, allocated disable threshold: 0.9
2016-10-04 10:16:13,750 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Deployment found - P0=VM[User|i-14-2619-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(4)-Host(9)-Storage(Volume(3392|ROOT-->Pool(5))]
2016-10-04 10:16:13,791 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking SecurityGroupProvider to prepare for
Nic[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181]
2016-10-04 10:16:13,795 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking VirtualRouter to prepare for
Nic[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181]
2016-10-04 10:16:23,552 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Checking if we need to prepare 1 volumes for
VM[User|i-14-2619-VM]
2016-10-04 10:16:24,902 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:5] is
unreachable: Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
2016-10-04 10:16:24,904 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Cleaning up resources for the vm VM[User|i-14-2619-VM] in
Starting state
2016-10-04 10:16:24,906 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Seq 9-6168524114614034375: Sending { Cmd , MgmtId:
144344642087, via: 9(xenserver11), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-14-2619-VM","wait":0}}]
}
2016-10-04 10:16:24,907 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Seq 9-6168524114614034375: Executing: { Cmd , MgmtId:
144344642087, via: 9(xenserver11), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-14-2619-VM","wait":0}}]
}
2016-10-04 10:16:24,927 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking SecurityGroupProvider to release
NicProfile[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181-vlan://untagged
2016-10-04 10:16:24,927 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking VirtualRouter to release
NicProfile[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181-vlan://untagged
2016-10-04 10:16:24,927 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Successfully released network resources for the vm
VM[User|i-14-2619-VM]
2016-10-04 10:16:24,927 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Successfully cleanued up resources for the vm
VM[User|i-14-2619-VM] in Starting state
2016-10-04 10:16:24,983 DEBUG [c.c.s.StorageManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Checking pool: 6 for volume allocation
[Vol[3392|vm=2619|ROOT]], maxSize : 2389599846400, totalAllocatedSize :
1986715975680, askingSize : 53687091200, allocated disable threshold: 0.9
2016-10-04 10:16:24,985 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Deployment found - P0=VM[User|i-14-2619-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(5)-Host(10)-Storage(Volume(3392|ROOT-->Pool(6))]
2016-10-04 10:16:25,044 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking SecurityGroupProvider to prepare for
Nic[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181]
2016-10-04 10:16:25,048 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking VirtualRouter to prepare for
Nic[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181]
2016-10-04 10:16:34,725 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Checking if we need to prepare 1 volumes for
VM[User|i-14-2619-VM]
2016-10-04 10:16:36,145 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:6] is
unreachable: Unable to create Vol[3392|vm=2619|ROOT]:Catch Exception
com.cloud.utils.exception.CloudRuntimeException for template due to
com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to
the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to
mount FileSR. Errno is 32]
2016-10-04 10:16:36,147 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Cleaning up resources for the vm VM[User|i-14-2619-VM] in
Starting state
2016-10-04 10:16:36,150 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Seq 10-2442639847895082119: Sending { Cmd , MgmtId:
144344642087, via: 10(xenserver12), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-14-2619-VM","wait":0}}]
}
2016-10-04 10:16:36,150 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Seq 10-2442639847895082119: Executing: { Cmd , MgmtId:
144344642087, via: 10(xenserver12), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-14-2619-VM","wait":0}}]
}
2016-10-04 10:16:36,170 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking SecurityGroupProvider to release
NicProfile[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181-vlan://untagged
2016-10-04 10:16:36,171 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Asking VirtualRouter to release
NicProfile[2445-2619-df6d6e34-127a-456b-8f72-4dc1e5395ec1-192.168.100.181-vlan://untagged
2016-10-04 10:16:36,171 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Successfully released network resources for the vm
VM[User|i-14-2619-VM]
2016-10-04 10:16:36,171 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Successfully cleanued up resources for the vm
VM[User|i-14-2619-VM] in Starting state
2016-10-04 10:16:36,207 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Invocation exception, caused by:
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[User|i-14-2619-VM]Scope=interface com.cloud.dc.DataCenter;
id=1
2016-10-04 10:16:36,207 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044 ctx-bf62c9d5)
(logid:2c9e5bb6) Rethrow exception
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[User|i-14-2619-VM]Scope=interface com.cloud.dc.DataCenter;
id=1
2016-10-04 10:16:36,207 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-166:ctx-a6b8f69f job-32042/job-32044) (logid:2c9e5bb6) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 2619, job origin: 32042
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[User|i-14-2619-VM]Scope=interface com.cloud.dc.DataCenter;
id=1
2016-10-04 10:16:36,225 DEBUG [c.c.v.UserVmManagerImpl]
(API-Job-Executor-43:ctx-99780220 job-32042 ctx-a183e59f) (logid:2c9e5bb6)
Destroying vm VM[User|i-14-2619-VM] as it failed to create on Host with Id:null
2016-10-04 10:16:36,261 WARN [o.a.c.alerts] (API-Job-Executor-43:ctx-99780220
job-32042 ctx-a183e59f) (logid:2c9e5bb6) alertType:: 8 // dataCenterId:: 1 //
podId:: 1 // clusterId:: null // message:: Failed to deploy Vm with Id: 2619,
on Host with Id: null
Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to
create a deployment for VM[User|i-14-2619-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2016-10-04 10:16:50,274 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(catalina-exec-7:ctx-8a1c2103 ctx-c11e7fe3) (logid:883ff841) submit async
job-32047, details: AsyncJobVO {id:32047, userId: 2, accountId: 2,
instanceType: VirtualMachine, instanceId: 2619, cmd:
org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin, cmdInfo:
{"id":"c16b71be-0fa7-4994-ac2f-6d3f75d60343","response":"json","ctxDetails":"{\"interface
com.cloud.vm.VirtualMachine\":\"c16b71be-0fa7-4994-ac2f-6d3f75d60343\"}","cmdEventType":"VM.DESTROY","ctxUserId":"2","httpmethod":"GET","_":"1475565410315","projectid":"435d00e2-0487-4229-b06f-2a03c9552e81","uuid":"c16b71be-0fa7-4994-ac2f-6d3f75d60343","ctxAccountId":"2","expunge":"true","ctxStartEventId":"64508"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 144344642087, completeMsid: null, lastUpdated: null,
lastPolled: null, created: null}
2016-10-04 10:16:50,274 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047) (logid:4dd26cca) Executing
AsyncJobVO {id:32047, userId: 2, accountId: 2, instanceType: VirtualMachine,
instanceId: 2619, cmd:
org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin, cmdInfo:
{"id":"c16b71be-0fa7-4994-ac2f-6d3f75d60343","response":"json","ctxDetails":"{\"interface
com.cloud.vm.VirtualMachine\":\"c16b71be-0fa7-4994-ac2f-6d3f75d60343\"}","cmdEventType":"VM.DESTROY","ctxUserId":"2","httpmethod":"GET","_":"1475565410315","projectid":"435d00e2-0487-4229-b06f-2a03c9552e81","uuid":"c16b71be-0fa7-4994-ac2f-6d3f75d60343","ctxAccountId":"2","expunge":"true","ctxStartEventId":"64508"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 144344642087, completeMsid: null, lastUpdated: null,
lastPolled: null, created: null}
2016-10-04 10:16:50,297 DEBUG [c.c.v.VirtualMachineManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca)
Destroying vm VM[User|i-14-2619-VM]
2016-10-04 10:16:50,304 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca) Sync
job-32048 execution on object VmWorkJobQueue.2619
2016-10-04 10:16:51,459 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-1:ctx-984aa544 job-32047/job-32048) (logid:4dd26cca) Run VM
work job: com.cloud.vm.VmWorkStop for VM 2619, job origin: 32047
2016-10-04 10:16:51,460 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-1:ctx-984aa544 job-32047/job-32048 ctx-d8896007)
(logid:4dd26cca) Execute VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":2,"accountId":2,"vmId":2619,"handlerName":"VirtualMachineManagerImpl"}
2016-10-04 10:16:51,465 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-1:ctx-984aa544 job-32047/job-32048 ctx-d8896007)
(logid:4dd26cca) Stopped called on VM[User|i-14-2619-VM] but the state is Error
2016-10-04 10:16:51,465 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-1:ctx-984aa544 job-32047/job-32048 ctx-d8896007)
(logid:4dd26cca) Done executing VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":2,"accountId":2,"vmId":2619,"handlerName":"VirtualMachineManagerImpl"}
2016-10-04 10:16:51,471 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-1:ctx-984aa544 job-32047/job-32048) (logid:4dd26cca) Done
with run of VM work job: com.cloud.vm.VmWorkStop for VM 2619, job origin: 32047
2016-10-04 10:16:51,480 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca) Sync
job-32049 execution on object VmWorkJobQueue.2619
2016-10-04 10:16:53,460 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-4:ctx-41d6546d job-32047/job-32049) (logid:4dd26cca) Run VM
work job: com.cloud.vm.snapshot.VmWorkDeleteAllVMSnapshots for VM 2619, job
origin: 32047
2016-10-04 10:16:53,461 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-4:ctx-41d6546d job-32047/job-32049 ctx-c3507b09)
(logid:4dd26cca) Execute VM work job:
com.cloud.vm.snapshot.VmWorkDeleteAllVMSnapshots{"userId":2,"accountId":2,"vmId":2619,"handlerName":"VMSnapshotManagerImpl"}
2016-10-04 10:16:53,463 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-4:ctx-41d6546d job-32047/job-32049 ctx-c3507b09)
(logid:4dd26cca) Done executing VM work job:
com.cloud.vm.snapshot.VmWorkDeleteAllVMSnapshots{"userId":2,"accountId":2,"vmId":2619,"handlerName":"VMSnapshotManagerImpl"}
2016-10-04 10:16:53,468 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-4:ctx-41d6546d job-32047/job-32049) (logid:4dd26cca) Done
with run of VM work job: com.cloud.vm.snapshot.VmWorkDeleteAllVMSnapshots for
VM 2619, job origin: 32047
2016-10-04 10:16:53,493 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca) Sync
job-32050 execution on object VmWorkJobQueue.2619
2016-10-04 10:16:55,458 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-5:ctx-ddaee7a1 job-32047/job-32050) (logid:4dd26cca) Run VM
work job: com.cloud.vm.VmWorkStop for VM 2619, job origin: 32047
2016-10-04 10:16:55,459 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-5:ctx-ddaee7a1 job-32047/job-32050 ctx-4e795c24)
(logid:4dd26cca) Execute VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":2,"accountId":2,"vmId":2619,"handlerName":"VirtualMachineManagerImpl"}
2016-10-04 10:16:55,463 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-5:ctx-ddaee7a1 job-32047/job-32050 ctx-4e795c24)
(logid:4dd26cca) Stopped called on VM[User|i-14-2619-VM] but the state is
Expunging
2016-10-04 10:16:55,478 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-5:ctx-ddaee7a1 job-32047/job-32050 ctx-4e795c24)
(logid:4dd26cca) Done executing VM work job:
com.cloud.vm.VmWorkStop{"cleanup":false,"userId":2,"accountId":2,"vmId":2619,"handlerName":"VirtualMachineManagerImpl"}
2016-10-04 10:16:55,484 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-5:ctx-ddaee7a1 job-32047/job-32050) (logid:4dd26cca) Done
with run of VM work job: com.cloud.vm.VmWorkStop for VM 2619, job origin: 32047
2016-10-04 10:16:55,496 DEBUG [c.c.v.VirtualMachineManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca)
Destroying vm VM[User|i-14-2619-VM]
2016-10-04 10:16:55,496 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca)
Cleaning network for vm: 2619
2016-10-04 10:16:55,537 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca)
Cleaning storage for vm: 2619
2016-10-04 10:16:55,540 DEBUG [c.c.v.VirtualMachineManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca)
Expunged VM[User|i-14-2619-VM]
2016-10-04 10:16:55,540 DEBUG [c.c.v.UserVmManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca)
Starting cleaning up vm VM[User|i-14-2619-VM] resources...
2016-10-04 10:16:55,543 INFO [c.c.n.s.SecurityGroupManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca)
Disassociated 1 network groups from uservm 2619
2016-10-04 10:16:55,545 DEBUG [c.c.n.s.SecurityGroupManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca)
Security group mappings are removed successfully for vm id=2619
2016-10-04 10:16:55,552 DEBUG [c.c.n.f.FirewallManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca) No
firewall rules are found for vm id=2619
2016-10-04 10:16:55,556 DEBUG [c.c.v.UserVmManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca)
Firewall rules are removed successfully as a part of vm id=2619 expunge
2016-10-04 10:16:55,558 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca) No
port forwarding rules are found for vm id=2619
2016-10-04 10:16:55,558 DEBUG [c.c.v.UserVmManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca) Port
forwarding rules are removed successfully as a part of vm id=2619 expunge
2016-10-04 10:16:55,559 DEBUG [c.c.v.UserVmManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca)
Removed vm id=2619 from all load balancers as a part of expunge process
2016-10-04 10:16:55,560 DEBUG [c.c.v.UserVmManagerImpl]
(API-Job-Executor-46:ctx-530f884a job-32047 ctx-6893caee) (logid:4dd26cca)
Successfully cleaned up vm VM[User|i-14-2619-VM] resources as a part of expunge
process
2016-10-04 10:28:22,293 DEBUG [c.c.a.ApiServlet] (catalina-exec-1:ctx-3b446b97)
(logid:cbad9140) ===START=== 172.16.1.80 -- GET
command=listVirtualMachines&response=json&projectid=435d00e2-0487-4229-b06f-2a03c9552e81&keyword=red&listAll=true&page=1&pagesize=20&_=1475566102619
2016-10-04 10:28:22,334 DEBUG [c.c.a.ApiServlet] (catalina-exec-1:ctx-3b446b97
ctx-b07d758e) (logid:cbad9140) ===END=== 172.16.1.80 -- GET
command=listVirtualMachines&response=json&projectid=435d00e2-0487-4229-b06f-2a03c9552e81&keyword=red&listAll=true&page=1&pagesize=20&_=1475566102619
