[
https://issues.apache.org/jira/browse/CLOUDSTACK-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sudha Ponnaganti updated CLOUDSTACK-5665:
-----------------------------------------
Description:
XEN hotfix installation procedure
1. Hosts master slave installed with
XS 6.0.2 base + Hotfix XS602E001 to XS602E029 + CSPs
2. Basic zone with cluster of above master & slave hosts
security group default with NO security rules
Deploy two VMs
3. MS: master host put to maintenance mode - VMs migrate to slave host
4. MS: unmanage cluster
5. master host: install hotfix XS602E030 + CSP , reboot master
6. MS: master host cancel maintenance mode
7. MS: manage cluster
8. MS: slave host put to maintenance mode, VMs FAIL migrate to master
Unable to complete hotfix installation
2013-12-26 16:51:25,307 INFO [xen.resource.CitrixResourceBase]
(DirectAgent-56:null) VM does not exist on
XenServerc6c75483-b35b-4a7c-9e2d-2274e34dd437
2013-12-26 16:51:25,307 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-56:null) Seq 1-265617550: Response Received:
2013-12-26 16:51:25,307 DEBUG [agent.transport.Request] (DirectAgent-56:null)
Seq 1-265617550: Processing: { Ans: , MgmtId: 7692017993539, via: 1, Ver: v1,
Flags: 110, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM
does not exist","wait":0}}] }
2013-12-26 16:51:25,307 DEBUG [agent.manager.AgentAttache]
(DirectAgent-56:null) Seq 1-265617550: Unable to find listener.
2013-12-26 16:51:25,307 DEBUG [agent.manager.AgentAttache]
(DirectAgent-56:null) Seq 1-265617550: No more commands found
2013-12-26 16:51:25,313 DEBUG [vm.dao.VMInstanceDaoImpl] (HA-Worker-3:work-19)
Unable to update VM[DomainRouter|r-4-VM]: DB Data={Host=4; State=Running;
updated=13; time=Thu Dec 26 16:51:25 PST 2013} New Data: {Host=4;
State=Stopping; updated=12; time=Thu Dec 26 16:51:25 PST 2013} Stale Data:
{Host=4; State=Running; updated=11; time=Thu Dec 26 16:51:22 PST 2013}
2013-12-26 16:51:25,313 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(HA-Worker-3:work-19) Unable to stop VM due to VM is being operated on.
2013-12-26 16:51:25,313 WARN [cloud.ha.HighAvailabilityManagerImpl]
(HA-Worker-3:work-19) Unable to migrate vm from 4
2013-12-26 16:51:25,314 DEBUG [cloud.resource.ResourceManagerImpl]
(HA-Worker-3:work-19) No next resource state for host 4 while current state is
ErrorInMaintenance with event UnableToMigrate
com.cloud.utils.fsm.NoTransitionException: No next resource state found for
current state =ErrorInMaintenance event =UnableToMigrate
at
com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1178)
at
com.cloud.resource.ResourceManagerImpl.maintenanceFailed(ResourceManagerImpl.java:2313)
at
com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:602)
at
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:858)
2013-12-26 16:51:25,314 INFO [cloud.ha.HighAvailabilityManagerImpl]
(HA-Worker-3:work-19) Completed HAWork[19-Migration-4-Running-Migrating]
2013-12-26 16:51:25,324 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-57:null) Task failed! Task record: uuid:
38ce6cd4-9ab2-7e46-4d10-dafe25a83084
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Thu Dec 26 16:55:03 PST 2013
finished: Thu Dec 26 16:55:03 PST 2013
status: failure
residentOn: com.xensource.xenapi.Host@40561b9c
progress: 1.0
type: <none/>
result:
errorInfo: [VM_REQUIRES_SR,
OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2013-12-26 16:51:25,332 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-57:null) Unable to migrate VM(s-1-VM) from
host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed! Task record:
uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Thu Dec 26 16:55:03 PST 2013
finished: Thu Dec 26 16:55:03 PST 2013
status: failure
residentOn: com.xensource.xenapi.Host@40561b9c
progress: 1.0
type: <none/>
result:
errorInfo: [VM_REQUIRES_SR,
OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
Task failed! Task record: uuid:
38ce6cd4-9ab2-7e46-4d10-dafe25a83084
nameLabel: Async.VM.pool_migrate
nameDescription:
:
residentOn: com.xensource.xenapi.Host@40561b9c
progress: 1.0
type: <none/>
result:
errorInfo: [VM_REQUIRES_SR,
OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3789)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3346)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:511)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-12-26 16:51:25,342 DEBUG [xen.resource.CitrixResourceBase]
(DirectAgent-57:null) 6. The VM s-1-VM is in Migrating state
2013-12-26 16:51:25,342 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-57:null) Seq 4-961806364: Response Received:
2013-12-26 16:51:25,342 DEBUG [agent.transport.Request] (DirectAgent-57:null)
Seq 4-961806364: Processing: { Ans: , MgmtId: 7692017993539, via: 4, Ver: v1,
Flags: 110,
[{"com.cloud.agent.api.MigrateAnswer":{"result":false,"details":"Catch
Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due
to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
VM(s-1-VM) from host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed!
Task record: uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084\n
nameLabel: Async.VM.pool_migrate\n nameDescription: \n
allowedOperations: []\n currentOperations: {}\n created: Thu Dec
26 16:55:03 PST 2013\n finished: Thu Dec 26 16:55:03 PST 2013\n
status: failure\n residentOn:
com.xensource.xenapi.Host@40561b9c\n progress: 1.0\n
type: <none/>\n result: \n errorInfo: [VM_REQUIRES_SR,
OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]\n otherConfig: {}\n
subtaskOf: com.xensource.xenapi.Task@aaf13f6f\n subtasks:
[]\n","wait":0}}] }
2013-12-26 16:51:25,342 DEBUG [agent.transport.Request] (HA-Worker-0:work-16)
Seq 4-961806364: Received: { Ans: , MgmtId: 7692017993539, via: 4, Ver: v1,
Flags: 110, { MigrateAnswer } }
2013-12-26 16:51:25,342 ERROR [cloud.vm.VirtualMachineManagerImpl]
(HA-Worker-0:work-16) Unable to migrate due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException: Migration failed due to
com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(s-1-VM)
from host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed! Task
record: uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Thu Dec 26 16:55:03 PST 2013
finished: Thu Dec 26 16:55:03 PST 2013
status: failure
residentOn: com.xensource.xenapi.Host@40561b9c
progress: 1.0
type: <none/>
result:
errorInfo: [VM_REQUIRES_SR,
OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2013-12-26 16:51:25,342 INFO [cloud.vm.VirtualMachineManagerImpl]
(HA-Worker-0:work-16) Migration was unsuccessful. Cleaning up:
VM[SecondaryStorageVm|s-1-VM]
2013-12-26 16:51:25,346 DEBUG [agent.manager.AgentAttache]
(DirectAgent-57:null) Seq 4-961806364: No more commands found
2013-12-26 16:51:25,355 WARN [apache.cloudstack.alerts] (HA-Worker-0:work-16)
alertType:: 17 // dataCenterId:: 1 // podId:: 1 // clusterId:: null //
message:: Unable to migrate vm s-1-VM from host Rack2Host19.lab.vmops.com in
zone z1 and pod z1p1
2013-12-26 16:51:25,356 DEBUG [cloud.alert.AlertManagerImpl]
(HA-Worker-0:work-16) Have already sent: 1 emails for alert type '17' --
skipping send email
2013-12-26 16:51:25,357 DEBUG [agent.manager.AgentAttache]
(HA-Worker-0:work-16) Seq 1-265617551: Routed from 7692017993539
2013-12-26 16:51:25,358 DEBUG [agent.transport.Request] (HA-Worker-0:work-16)
Seq 1-265617551: Sending { Cmd , MgmtId: 7692017993539, via: 1, Ver: v1,
Flags: 100111,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"s-1-VM","wait":0}}]
}
2013-12-26 16:51:25,358 DEBUG [agent.transport.Request] (HA-Worker-0:work-16)
Seq 1-265617551: Executing: { Cmd , MgmtId: 7692017993539, via: 1, Ver: v1,
Flags: 100111,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"s-1-VM","wait":0}}]
}
2013-12-26 16:51:25,358 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-62:null) Seq 1-265617551: Executing request
2013-12-26 16:51:25,368 DEBUG [cloud.capacity.CapacityManagerImpl]
(HA-Worker-0:work-16) VM state transitted from :Migrating to Running with
event: OperationFailedvm's original host id: 4 new host id: 4 host id before
state transition: 1
2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl]
(HA-Worker-0:work-16) Hosts's actual total CPU: 9044 and CPU after applying
overprovisioning: 9044
2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl]
(HA-Worker-0:work-16) Hosts's actual total RAM: 15988871808 and RAM after
applying overprovisioning: 15988872192
2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl]
(HA-Worker-0:work-16) release cpu from host: 1, old used: 500,reserved: 0,
actual total: 9044, total with overprovisioning: 9044; new used: 0,reserved:0;
movedfromreserved: false,moveToReserveredfalse
2013-12-26 16:51:25,376 DEBUG [cloud.capacity.CapacityManagerImpl]
(HA-Worker-0:work-16) release mem from host: 1, old used: 268435456,reserved:
0, total: 15988872192; new used: 0,reserved:0; movedfromreserved:
false,moveToReserveredfalse
2013-12-26 16:51:25,385 INFO [xen.resource.CitrixResourceBase]
(DirectAgent-62:null) VM does not exist on
XenServerc6c75483-b35b-4a7c-9e2d-2274e34dd437
2013-12-26 16:51:25,385 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-62:null) Seq 1-265617551: Response Received:
2013-12-26 16:51:25,386 DEBUG [agent.transport.Request] (DirectAgent-62:null)
Seq 1-265617551: Processing: { Ans: , MgmtId: 7692017993539, via: 1, Ver: v1,
Flags: 110, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM
does not exist","wait":0}}] }
2013-12-26 16:51:25,386 DEBUG [agent.manager.AgentAttache]
(DirectAgent-62:null) Seq 1-265617551: Unable to find listener.
2013-12-26 16:51:25,386 DEBUG [agent.manager.AgentAttache]
(DirectAgent-62:null) Seq 1-265617551: No more commands found
2013-12-26 16:51:25,390 DEBUG [vm.dao.VMInstanceDaoImpl] (HA-Worker-0:work-16)
Unable to update VM[SecondaryStorageVm|s-1-VM]: DB Data={Host=4; State=Running;
updated=13; time=Thu Dec 26 16:51:25 PST 2013} New Data: {Host=4;
State=Stopping; updated=12; time=Thu Dec 26 16:51:25 PST 2013} Stale Data:
{Host=4; State=Running; updated=11; time=Thu Dec 26 16:51:23 PST 2013}
2013-12-26 16:51:25,390 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(HA-Worker-0:work-16) Unable to stop VM due to VM is being operated on.
2013-12-26 16:51:25,390 WARN [cloud.ha.HighAvailabilityManagerImpl]
(HA-Worker-0:work-16) Unable to migrate vm from 4
2013-12-26 16:51:25,391 DEBUG [cloud.resource.ResourceManagerImpl]
(HA-Worker-0:work-16) No next resource state for host 4 while current state is
ErrorInMaintenance with event UnableToMigrate
com.cloud.utils.fsm.NoTransitionException: No next resource state found for
current state =ErrorInMaintenance event =UnableToMigrate
at
com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1178)
at
com.cloud.resource.ResourceManagerImpl.maintenanceFailed(ResourceManagerImpl.java:2313)
at
com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:602)
at
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:858)
2013-12-26 16:51:25,392 INFO [cloud.ha.HighAvailabilityManagerImpl]
(HA-Worker-0:work-16) Completed HAWork[16-Migration-1-Running-Migrating]
2013-12-26 16:51:26,339 DEBUG [agent.manager.AgentManagerImpl]
(AgentManager-Handler-10:null) SeqA 2-416: Processing Seq 2-416: { Cmd ,
MgmtId: -1, via: 2, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2013-12-26 16:51:26,346 DEBUG [agent.manager.AgentManagerImpl]
(AgentManager-Handler-10:null) SeqA 2-416: Sending Seq 2-416: { Ans: , MgmtId:
7692017993539, via: 2, Ver: v1, Flags: 100010, [{"com.cloud.agent.ap
was:
http://wiki-ccp.citrix.com/pages/viewpage.action?pageId=11829631 :
XEN hotfix installation procedure
1. Hosts master slave installed with
XS 6.0.2 base + Hotfix XS602E001 to XS602E029 + CSPs
2. Basic zone with cluster of above master & slave hosts
security group default with NO security rules
Deploy two VMs
3. MS: master host put to maintenance mode - VMs migrate to slave host
4. MS: unmanage cluster
5. master host: install hotfix XS602E030 + CSP , reboot master
6. MS: master host cancel maintenance mode
7. MS: manage cluster
8. MS: slave host put to maintenance mode, VMs FAIL migrate to master
Unable to complete hotfix installation
2013-12-26 16:51:25,307 INFO [xen.resource.CitrixResourceBase]
(DirectAgent-56:null) VM does not exist on
XenServerc6c75483-b35b-4a7c-9e2d-2274e34dd437
2013-12-26 16:51:25,307 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-56:null) Seq 1-265617550: Response Received:
2013-12-26 16:51:25,307 DEBUG [agent.transport.Request] (DirectAgent-56:null)
Seq 1-265617550: Processing: { Ans: , MgmtId: 7692017993539, via: 1, Ver: v1,
Flags: 110, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM
does not exist","wait":0}}] }
2013-12-26 16:51:25,307 DEBUG [agent.manager.AgentAttache]
(DirectAgent-56:null) Seq 1-265617550: Unable to find listener.
2013-12-26 16:51:25,307 DEBUG [agent.manager.AgentAttache]
(DirectAgent-56:null) Seq 1-265617550: No more commands found
2013-12-26 16:51:25,313 DEBUG [vm.dao.VMInstanceDaoImpl] (HA-Worker-3:work-19)
Unable to update VM[DomainRouter|r-4-VM]: DB Data={Host=4; State=Running;
updated=13; time=Thu Dec 26 16:51:25 PST 2013} New Data: {Host=4;
State=Stopping; updated=12; time=Thu Dec 26 16:51:25 PST 2013} Stale Data:
{Host=4; State=Running; updated=11; time=Thu Dec 26 16:51:22 PST 2013}
2013-12-26 16:51:25,313 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(HA-Worker-3:work-19) Unable to stop VM due to VM is being operated on.
2013-12-26 16:51:25,313 WARN [cloud.ha.HighAvailabilityManagerImpl]
(HA-Worker-3:work-19) Unable to migrate vm from 4
2013-12-26 16:51:25,314 DEBUG [cloud.resource.ResourceManagerImpl]
(HA-Worker-3:work-19) No next resource state for host 4 while current state is
ErrorInMaintenance with event UnableToMigrate
com.cloud.utils.fsm.NoTransitionException: No next resource state found for
current state =ErrorInMaintenance event =UnableToMigrate
at
com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1178)
at
com.cloud.resource.ResourceManagerImpl.maintenanceFailed(ResourceManagerImpl.java:2313)
at
com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:602)
at
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:858)
2013-12-26 16:51:25,314 INFO [cloud.ha.HighAvailabilityManagerImpl]
(HA-Worker-3:work-19) Completed HAWork[19-Migration-4-Running-Migrating]
2013-12-26 16:51:25,324 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-57:null) Task failed! Task record: uuid:
38ce6cd4-9ab2-7e46-4d10-dafe25a83084
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Thu Dec 26 16:55:03 PST 2013
finished: Thu Dec 26 16:55:03 PST 2013
status: failure
residentOn: com.xensource.xenapi.Host@40561b9c
progress: 1.0
type: <none/>
result:
errorInfo: [VM_REQUIRES_SR,
OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2013-12-26 16:51:25,332 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-57:null) Unable to migrate VM(s-1-VM) from
host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed! Task record:
uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Thu Dec 26 16:55:03 PST 2013
finished: Thu Dec 26 16:55:03 PST 2013
status: failure
residentOn: com.xensource.xenapi.Host@40561b9c
progress: 1.0
type: <none/>
result:
errorInfo: [VM_REQUIRES_SR,
OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
Task failed! Task record: uuid:
38ce6cd4-9ab2-7e46-4d10-dafe25a83084
nameLabel: Async.VM.pool_migrate
nameDescription:
:
residentOn: com.xensource.xenapi.Host@40561b9c
progress: 1.0
type: <none/>
result:
errorInfo: [VM_REQUIRES_SR,
OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3789)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3346)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:511)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-12-26 16:51:25,342 DEBUG [xen.resource.CitrixResourceBase]
(DirectAgent-57:null) 6. The VM s-1-VM is in Migrating state
2013-12-26 16:51:25,342 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-57:null) Seq 4-961806364: Response Received:
2013-12-26 16:51:25,342 DEBUG [agent.transport.Request] (DirectAgent-57:null)
Seq 4-961806364: Processing: { Ans: , MgmtId: 7692017993539, via: 4, Ver: v1,
Flags: 110,
[{"com.cloud.agent.api.MigrateAnswer":{"result":false,"details":"Catch
Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due
to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
VM(s-1-VM) from host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed!
Task record: uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084\n
nameLabel: Async.VM.pool_migrate\n nameDescription: \n
allowedOperations: []\n currentOperations: {}\n created: Thu Dec
26 16:55:03 PST 2013\n finished: Thu Dec 26 16:55:03 PST 2013\n
status: failure\n residentOn:
com.xensource.xenapi.Host@40561b9c\n progress: 1.0\n
type: <none/>\n result: \n errorInfo: [VM_REQUIRES_SR,
OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]\n otherConfig: {}\n
subtaskOf: com.xensource.xenapi.Task@aaf13f6f\n subtasks:
[]\n","wait":0}}] }
2013-12-26 16:51:25,342 DEBUG [agent.transport.Request] (HA-Worker-0:work-16)
Seq 4-961806364: Received: { Ans: , MgmtId: 7692017993539, via: 4, Ver: v1,
Flags: 110, { MigrateAnswer } }
2013-12-26 16:51:25,342 ERROR [cloud.vm.VirtualMachineManagerImpl]
(HA-Worker-0:work-16) Unable to migrate due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException: Migration failed due to
com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(s-1-VM)
from host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed! Task
record: uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Thu Dec 26 16:55:03 PST 2013
finished: Thu Dec 26 16:55:03 PST 2013
status: failure
residentOn: com.xensource.xenapi.Host@40561b9c
progress: 1.0
type: <none/>
result:
errorInfo: [VM_REQUIRES_SR,
OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2013-12-26 16:51:25,342 INFO [cloud.vm.VirtualMachineManagerImpl]
(HA-Worker-0:work-16) Migration was unsuccessful. Cleaning up:
VM[SecondaryStorageVm|s-1-VM]
2013-12-26 16:51:25,346 DEBUG [agent.manager.AgentAttache]
(DirectAgent-57:null) Seq 4-961806364: No more commands found
2013-12-26 16:51:25,355 WARN [apache.cloudstack.alerts] (HA-Worker-0:work-16)
alertType:: 17 // dataCenterId:: 1 // podId:: 1 // clusterId:: null //
message:: Unable to migrate vm s-1-VM from host Rack2Host19.lab.vmops.com in
zone z1 and pod z1p1
2013-12-26 16:51:25,356 DEBUG [cloud.alert.AlertManagerImpl]
(HA-Worker-0:work-16) Have already sent: 1 emails for alert type '17' --
skipping send email
2013-12-26 16:51:25,357 DEBUG [agent.manager.AgentAttache]
(HA-Worker-0:work-16) Seq 1-265617551: Routed from 7692017993539
2013-12-26 16:51:25,358 DEBUG [agent.transport.Request] (HA-Worker-0:work-16)
Seq 1-265617551: Sending { Cmd , MgmtId: 7692017993539, via: 1, Ver: v1,
Flags: 100111,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"s-1-VM","wait":0}}]
}
2013-12-26 16:51:25,358 DEBUG [agent.transport.Request] (HA-Worker-0:work-16)
Seq 1-265617551: Executing: { Cmd , MgmtId: 7692017993539, via: 1, Ver: v1,
Flags: 100111,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"s-1-VM","wait":0}}]
}
2013-12-26 16:51:25,358 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-62:null) Seq 1-265617551: Executing request
2013-12-26 16:51:25,368 DEBUG [cloud.capacity.CapacityManagerImpl]
(HA-Worker-0:work-16) VM state transitted from :Migrating to Running with
event: OperationFailedvm's original host id: 4 new host id: 4 host id before
state transition: 1
2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl]
(HA-Worker-0:work-16) Hosts's actual total CPU: 9044 and CPU after applying
overprovisioning: 9044
2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl]
(HA-Worker-0:work-16) Hosts's actual total RAM: 15988871808 and RAM after
applying overprovisioning: 15988872192
2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl]
(HA-Worker-0:work-16) release cpu from host: 1, old used: 500,reserved: 0,
actual total: 9044, total with overprovisioning: 9044; new used: 0,reserved:0;
movedfromreserved: false,moveToReserveredfalse
2013-12-26 16:51:25,376 DEBUG [cloud.capacity.CapacityManagerImpl]
(HA-Worker-0:work-16) release mem from host: 1, old used: 268435456,reserved:
0, total: 15988872192; new used: 0,reserved:0; movedfromreserved:
false,moveToReserveredfalse
2013-12-26 16:51:25,385 INFO [xen.resource.CitrixResourceBase]
(DirectAgent-62:null) VM does not exist on
XenServerc6c75483-b35b-4a7c-9e2d-2274e34dd437
2013-12-26 16:51:25,385 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-62:null) Seq 1-265617551: Response Received:
2013-12-26 16:51:25,386 DEBUG [agent.transport.Request] (DirectAgent-62:null)
Seq 1-265617551: Processing: { Ans: , MgmtId: 7692017993539, via: 1, Ver: v1,
Flags: 110, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM
does not exist","wait":0}}] }
2013-12-26 16:51:25,386 DEBUG [agent.manager.AgentAttache]
(DirectAgent-62:null) Seq 1-265617551: Unable to find listener.
2013-12-26 16:51:25,386 DEBUG [agent.manager.AgentAttache]
(DirectAgent-62:null) Seq 1-265617551: No more commands found
2013-12-26 16:51:25,390 DEBUG [vm.dao.VMInstanceDaoImpl] (HA-Worker-0:work-16)
Unable to update VM[SecondaryStorageVm|s-1-VM]: DB Data={Host=4; State=Running;
updated=13; time=Thu Dec 26 16:51:25 PST 2013} New Data: {Host=4;
State=Stopping; updated=12; time=Thu Dec 26 16:51:25 PST 2013} Stale Data:
{Host=4; State=Running; updated=11; time=Thu Dec 26 16:51:23 PST 2013}
2013-12-26 16:51:25,390 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(HA-Worker-0:work-16) Unable to stop VM due to VM is being operated on.
2013-12-26 16:51:25,390 WARN [cloud.ha.HighAvailabilityManagerImpl]
(HA-Worker-0:work-16) Unable to migrate vm from 4
2013-12-26 16:51:25,391 DEBUG [cloud.resource.ResourceManagerImpl]
(HA-Worker-0:work-16) No next resource state for host 4 while current state is
ErrorInMaintenance with event UnableToMigrate
com.cloud.utils.fsm.NoTransitionException: No next resource state found for
current state =ErrorInMaintenance event =UnableToMigrate
at
com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1178)
at
com.cloud.resource.ResourceManagerImpl.maintenanceFailed(ResourceManagerImpl.java:2313)
at
com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:602)
at
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:858)
2013-12-26 16:51:25,392 INFO [cloud.ha.HighAvailabilityManagerImpl]
(HA-Worker-0:work-16) Completed HAWork[16-Migration-1-Running-Migrating]
2013-12-26 16:51:26,339 DEBUG [agent.manager.AgentManagerImpl]
(AgentManager-Handler-10:null) SeqA 2-416: Processing Seq 2-416: { Cmd ,
MgmtId: -1, via: 2, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2013-12-26 16:51:26,346 DEBUG [agent.manager.AgentManagerImpl]
(AgentManager-Handler-10:null) SeqA 2-416: Sending Seq 2-416: { Ans: , MgmtId:
7692017993539, via: 2, Ver: v1, Flags: 100010, [{"com.cloud.agent.ap
Environment: (was: MS CloudPlatform-4.2.1-2-rhel6.3.tar.gz
host 10.223.51.3 XS 6.0.2 base + HF XS602E001 to XS602E029 + CSPs
10.223.51.4 XS 6.0.2 base + HF XS602E001 to XS602E029 + CSPs )
> XEN patch/hotfix certification - XS 6.0.2 XS602E030 patch installation fail
> ---------------------------------------------------------------------------
>
> Key: CLOUDSTACK-5665
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5665
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Management Server
> Affects Versions: 4.2.1
> Reporter: angeline shen
> Priority: Critical
> Fix For: 4.2.1
>
> Attachments: management-server.log.gz
>
>
>
> XEN hotfix installation procedure
> 1. Hosts master slave installed with
> XS 6.0.2 base + Hotfix XS602E001 to XS602E029 + CSPs
> 2. Basic zone with cluster of above master & slave hosts
> security group default with NO security rules
> Deploy two VMs
> 3. MS: master host put to maintenance mode - VMs migrate to slave host
> 4. MS: unmanage cluster
> 5. master host: install hotfix XS602E030 + CSP , reboot master
> 6. MS: master host cancel maintenance mode
> 7. MS: manage cluster
> 8. MS: slave host put to maintenance mode, VMs FAIL migrate to master
> Unable to complete hotfix installation
> 2013-12-26 16:51:25,307 INFO [xen.resource.CitrixResourceBase]
> (DirectAgent-56:null) VM does not exist on
> XenServerc6c75483-b35b-4a7c-9e2d-2274e34dd437
> 2013-12-26 16:51:25,307 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-56:null) Seq 1-265617550: Response Received:
> 2013-12-26 16:51:25,307 DEBUG [agent.transport.Request] (DirectAgent-56:null)
> Seq 1-265617550: Processing: { Ans: , MgmtId: 7692017993539, via: 1, Ver:
> v1, Flags: 110,
> [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM does not
> exist","wait":0}}] }
> 2013-12-26 16:51:25,307 DEBUG [agent.manager.AgentAttache]
> (DirectAgent-56:null) Seq 1-265617550: Unable to find listener.
> 2013-12-26 16:51:25,307 DEBUG [agent.manager.AgentAttache]
> (DirectAgent-56:null) Seq 1-265617550: No more commands found
> 2013-12-26 16:51:25,313 DEBUG [vm.dao.VMInstanceDaoImpl]
> (HA-Worker-3:work-19) Unable to update VM[DomainRouter|r-4-VM]: DB
> Data={Host=4; State=Running; updated=13; time=Thu Dec 26 16:51:25 PST 2013}
> New Data: {Host=4; State=Stopping; updated=12; time=Thu Dec 26 16:51:25 PST
> 2013} Stale Data: {Host=4; State=Running; updated=11; time=Thu Dec 26
> 16:51:22 PST 2013}
> 2013-12-26 16:51:25,313 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (HA-Worker-3:work-19) Unable to stop VM due to VM is being operated on.
> 2013-12-26 16:51:25,313 WARN [cloud.ha.HighAvailabilityManagerImpl]
> (HA-Worker-3:work-19) Unable to migrate vm from 4
> 2013-12-26 16:51:25,314 DEBUG [cloud.resource.ResourceManagerImpl]
> (HA-Worker-3:work-19) No next resource state for host 4 while current state
> is ErrorInMaintenance with event UnableToMigrate
> com.cloud.utils.fsm.NoTransitionException: No next resource state found for
> current state =ErrorInMaintenance event =UnableToMigrate
> at
> com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1178)
> at
> com.cloud.resource.ResourceManagerImpl.maintenanceFailed(ResourceManagerImpl.java:2313)
> at
> com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:602)
> at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:858)
> 2013-12-26 16:51:25,314 INFO [cloud.ha.HighAvailabilityManagerImpl]
> (HA-Worker-3:work-19) Completed HAWork[19-Migration-4-Running-Migrating]
> 2013-12-26 16:51:25,324 WARN [xen.resource.CitrixResourceBase]
> (DirectAgent-57:null) Task failed! Task record: uuid:
> 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Thu Dec 26 16:55:03 PST 2013
> finished: Thu Dec 26 16:55:03 PST 2013
> status: failure
> residentOn: com.xensource.xenapi.Host@40561b9c
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_REQUIRES_SR,
> OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
> OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> 2013-12-26 16:51:25,332 WARN [xen.resource.CitrixResourceBase]
> (DirectAgent-57:null) Unable to migrate VM(s-1-VM) from
> host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed! Task record:
> uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Thu Dec 26 16:55:03 PST 2013
> finished: Thu Dec 26 16:55:03 PST 2013
> status: failure
> residentOn: com.xensource.xenapi.Host@40561b9c
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_REQUIRES_SR,
> OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
> OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> Task failed! Task record: uuid:
> 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> :
> residentOn: com.xensource.xenapi.Host@40561b9c
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_REQUIRES_SR,
> OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
> OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3789)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3346)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:511)
> at
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59)
> at
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-12-26 16:51:25,342 DEBUG [xen.resource.CitrixResourceBase]
> (DirectAgent-57:null) 6. The VM s-1-VM is in Migrating state
> 2013-12-26 16:51:25,342 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-57:null) Seq 4-961806364: Response Received:
> 2013-12-26 16:51:25,342 DEBUG [agent.transport.Request] (DirectAgent-57:null)
> Seq 4-961806364: Processing: { Ans: , MgmtId: 7692017993539, via: 4, Ver:
> v1, Flags: 110,
> [{"com.cloud.agent.api.MigrateAnswer":{"result":false,"details":"Catch
> Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
> VM(s-1-VM) from host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task
> failed! Task record: uuid:
> 38ce6cd4-9ab2-7e46-4d10-dafe25a83084\n nameLabel:
> Async.VM.pool_migrate\n nameDescription: \n allowedOperations: []\n
> currentOperations: {}\n created: Thu Dec 26 16:55:03 PST 2013\n
> finished: Thu Dec 26 16:55:03 PST 2013\n status:
> failure\n residentOn: com.xensource.xenapi.Host@40561b9c\n
> progress: 1.0\n type: <none/>\n result: \n
> errorInfo: [VM_REQUIRES_SR,
> OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
> OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]\n otherConfig: {}\n
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f\n subtasks:
> []\n","wait":0}}] }
> 2013-12-26 16:51:25,342 DEBUG [agent.transport.Request] (HA-Worker-0:work-16)
> Seq 4-961806364: Received: { Ans: , MgmtId: 7692017993539, via: 4, Ver: v1,
> Flags: 110, { MigrateAnswer } }
> 2013-12-26 16:51:25,342 ERROR [cloud.vm.VirtualMachineManagerImpl]
> (HA-Worker-0:work-16) Unable to migrate due to Catch Exception
> com.cloud.utils.exception.CloudRuntimeException: Migration failed due to
> com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(s-1-VM)
> from host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed! Task
> record: uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Thu Dec 26 16:55:03 PST 2013
> finished: Thu Dec 26 16:55:03 PST 2013
> status: failure
> residentOn: com.xensource.xenapi.Host@40561b9c
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_REQUIRES_SR,
> OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
> OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> 2013-12-26 16:51:25,342 INFO [cloud.vm.VirtualMachineManagerImpl]
> (HA-Worker-0:work-16) Migration was unsuccessful. Cleaning up:
> VM[SecondaryStorageVm|s-1-VM]
> 2013-12-26 16:51:25,346 DEBUG [agent.manager.AgentAttache]
> (DirectAgent-57:null) Seq 4-961806364: No more commands found
> 2013-12-26 16:51:25,355 WARN [apache.cloudstack.alerts]
> (HA-Worker-0:work-16) alertType:: 17 // dataCenterId:: 1 // podId:: 1 //
> clusterId:: null // message:: Unable to migrate vm s-1-VM from host
> Rack2Host19.lab.vmops.com in zone z1 and pod z1p1
> 2013-12-26 16:51:25,356 DEBUG [cloud.alert.AlertManagerImpl]
> (HA-Worker-0:work-16) Have already sent: 1 emails for alert type '17' --
> skipping send email
> 2013-12-26 16:51:25,357 DEBUG [agent.manager.AgentAttache]
> (HA-Worker-0:work-16) Seq 1-265617551: Routed from 7692017993539
> 2013-12-26 16:51:25,358 DEBUG [agent.transport.Request] (HA-Worker-0:work-16)
> Seq 1-265617551: Sending { Cmd , MgmtId: 7692017993539, via: 1, Ver: v1,
> Flags: 100111,
> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"s-1-VM","wait":0}}]
> }
> 2013-12-26 16:51:25,358 DEBUG [agent.transport.Request] (HA-Worker-0:work-16)
> Seq 1-265617551: Executing: { Cmd , MgmtId: 7692017993539, via: 1, Ver: v1,
> Flags: 100111,
> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"s-1-VM","wait":0}}]
> }
> 2013-12-26 16:51:25,358 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-62:null) Seq 1-265617551: Executing request
> 2013-12-26 16:51:25,368 DEBUG [cloud.capacity.CapacityManagerImpl]
> (HA-Worker-0:work-16) VM state transitted from :Migrating to Running with
> event: OperationFailedvm's original host id: 4 new host id: 4 host id before
> state transition: 1
> 2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl]
> (HA-Worker-0:work-16) Hosts's actual total CPU: 9044 and CPU after applying
> overprovisioning: 9044
> 2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl]
> (HA-Worker-0:work-16) Hosts's actual total RAM: 15988871808 and RAM after
> applying overprovisioning: 15988872192
> 2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl]
> (HA-Worker-0:work-16) release cpu from host: 1, old used: 500,reserved: 0,
> actual total: 9044, total with overprovisioning: 9044; new used:
> 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2013-12-26 16:51:25,376 DEBUG [cloud.capacity.CapacityManagerImpl]
> (HA-Worker-0:work-16) release mem from host: 1, old used: 268435456,reserved:
> 0, total: 15988872192; new used: 0,reserved:0; movedfromreserved:
> false,moveToReserveredfalse
> 2013-12-26 16:51:25,385 INFO [xen.resource.CitrixResourceBase]
> (DirectAgent-62:null) VM does not exist on
> XenServerc6c75483-b35b-4a7c-9e2d-2274e34dd437
> 2013-12-26 16:51:25,385 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-62:null) Seq 1-265617551: Response Received:
> 2013-12-26 16:51:25,386 DEBUG [agent.transport.Request] (DirectAgent-62:null)
> Seq 1-265617551: Processing: { Ans: , MgmtId: 7692017993539, via: 1, Ver:
> v1, Flags: 110,
> [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM does not
> exist","wait":0}}] }
> 2013-12-26 16:51:25,386 DEBUG [agent.manager.AgentAttache]
> (DirectAgent-62:null) Seq 1-265617551: Unable to find listener.
> 2013-12-26 16:51:25,386 DEBUG [agent.manager.AgentAttache]
> (DirectAgent-62:null) Seq 1-265617551: No more commands found
> 2013-12-26 16:51:25,390 DEBUG [vm.dao.VMInstanceDaoImpl]
> (HA-Worker-0:work-16) Unable to update VM[SecondaryStorageVm|s-1-VM]: DB
> Data={Host=4; State=Running; updated=13; time=Thu Dec 26 16:51:25 PST 2013}
> New Data: {Host=4; State=Stopping; updated=12; time=Thu Dec 26 16:51:25 PST
> 2013} Stale Data: {Host=4; State=Running; updated=11; time=Thu Dec 26
> 16:51:23 PST 2013}
> 2013-12-26 16:51:25,390 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (HA-Worker-0:work-16) Unable to stop VM due to VM is being operated on.
> 2013-12-26 16:51:25,390 WARN [cloud.ha.HighAvailabilityManagerImpl]
> (HA-Worker-0:work-16) Unable to migrate vm from 4
> 2013-12-26 16:51:25,391 DEBUG [cloud.resource.ResourceManagerImpl]
> (HA-Worker-0:work-16) No next resource state for host 4 while current state
> is ErrorInMaintenance with event UnableToMigrate
> com.cloud.utils.fsm.NoTransitionException: No next resource state found for
> current state =ErrorInMaintenance event =UnableToMigrate
> at
> com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1178)
> at
> com.cloud.resource.ResourceManagerImpl.maintenanceFailed(ResourceManagerImpl.java:2313)
> at
> com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:602)
> at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:858)
> 2013-12-26 16:51:25,392 INFO [cloud.ha.HighAvailabilityManagerImpl]
> (HA-Worker-0:work-16) Completed HAWork[16-Migration-1-Running-Migrating]
> 2013-12-26 16:51:26,339 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-10:null) SeqA 2-416: Processing Seq 2-416: { Cmd ,
> MgmtId: -1, via: 2, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2013-12-26 16:51:26,346 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-10:null) SeqA 2-416: Sending Seq 2-416: { Ans: ,
> MgmtId: 7692017993539, via: 2, Ver: v1, Flags: 100010, [{"com.cloud.agent.ap
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
