Sangeetha Hariharan created CLOUDSTACK-2140:
-----------------------------------------------
Summary: Host is still marked as being in "Up" state when the host
is shutdown (when there are no more hosts in the cluster)
Key: CLOUDSTACK-2140
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2140
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Affects Versions: 4.2.0
Environment: build from master
Reporter: Sangeetha Hariharan
Fix For: 4.2.0
Host is still marked as being in "Up" state when the host is shutdown (when
there are no more hosts in the cluster.
Set up:
Advanced zone.
3 hosts in a cluster ( in my case host id - 7 ,8 ,9 ).
I did not have any problems when host 8 and host 9 where shutdown.
When I tried to shutdown host 7 , I see the host still being in "Up" state ,
even after the management server detected that it is not able to connect with
this host.
Following exception seen in management server logs:
2013-04-22 14:48:18,350 DEBUG [xen.resource.XenServerConnectionPool]
(DirectAgent-350:null) localLogout has problem Failed to read server's
response: connect timed out
2013-04-22 14:48:18,350 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-350:null) Unable to stop i-3-45-VM due to
com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of
slave 10.223.59.4 to 10.223.59.2 due to org.apache.xmlrpc.XmlRpcException:
Failed to read server's response: connect timed out
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:443)
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:661)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5583)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3728)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:474)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-04-22 14:48:18,364 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-350:null) Seq 9-72160431: Response Received:
2013-04-22 14:48:18,370 DEBUG [agent.transport.Request] (DirectAgent-350:null)
Seq 9-72160431: Processing: { Ans: , MgmtId: 7508777239729, via: 9, Ver: v1,
Flags: 110, [{"StopAnswer":{"result":false,"details":"Exception:
com.cloud.utils.exception.CloudRuntimeException\nMessage: Unable to reset
master of slave 10.223.59.4 to 10.223.59.2 due to
org.apache.xmlrpc.XmlRpcException: Failed to read server's response: connect
timed out\nStack: com.cloud.utils.exception.CloudRuntimeException: Unable to
reset master of slave 10.223.59.4 to 10.223.59.2 due to
org.apache.xmlrpc.XmlRpcException: Failed to read server's response: connect
timed out\n\tat
com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:443)\n\tat
com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:661)\n\tat
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5583)\n\tat
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3728)\n\tat
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:474)\n\tat
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)\n\tat
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)\n\tat
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)\n\tat
java.util.concurrent.FutureTask.run(FutureTask.java:166)\n\tat
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
java.lang.Thread.run(Thread.java:679)\n","wait":0}}] }
2013-04-22 14:48:18,370 DEBUG [agent.transport.Request] (DirectAgent-276:null)
Seq 9-72160431: Received: { Ans: , MgmtId: 7508777239729, via: 9, Ver: v1,
Flags: 110, { StopAnswer } }
2013-04-22 14:48:18,370 WARN [cloud.vm.VirtualMachineManagerImpl]
(DirectAgent-276:null) Unable to actually stop VM[User|anan5] but continue with
release because it's a force stop
2013-04-22 14:48:18,371 WARN [agent.manager.DirectAgentAttache]
(DirectAgent-276:null) Seq 7-1177944069: Exception caught
com.cloud.utils.exception.CloudRuntimeException: Unable to stop the virtual
machine due to Exception: com.cloud.utils.exception.CloudRuntimeException
Host entries in DB:
| 7 | Rack3Host17.lab.vmops.com |
ea8a5618-3e10-4a02-a6ea-9a8e10e7efc7 | Up | Routing |
10.223.59.2 | 255.255.255.192 | bc:30:5b:d4:1c:36 | 10.223.59.2
| 255.255.255.192 | bc:30:5b:d4:1c:36 | NULL | NULL
| NULL | 6 | 10.223.59.2 | 255.255.255.192 |
bc:30:5b:d4:1c:36 | NULL | 1 | 4 | 4 | 2261 |
iqn.2005-03.org.open-iscsi:7ad5ccd9c587 | NULL |
XenServer | 6.0.2 | 16190149248 |
com.cloud.hypervisor.xen.resource.XenServer602Resource | 4.2.0-SNAPSHOT | NULL
| NULL | xen-3.0-x86_64 ,
xen-3.0-x86_32p , hvm-3.0-x86_32 , hvm-3.0-x86_32p , hvm-3.0-x86_64 |
fb549fb6-82b6-4b74-a468-c511d744b238 | 1 |
1 | 0 | 1334307227 | 7508777239729 | 2013-04-19 00:20:22 |
2013-04-18 21:22:16 | NULL | 6 | Enabled | NULL | NULL
| Disabled |
| 8 | Rack3Host18.lab.vmops.com |
df1afd25-4871-41f7-bccb-44d8f9ce193d | Down | Routing |
10.223.59.3 | 255.255.255.192 | bc:30:5b:d4:23:54 | 10.223.59.3
| 255.255.255.192 | bc:30:5b:d4:23:54 | NULL | NULL
| NULL | 6 | 10.223.59.3 | 255.255.255.192 |
bc:30:5b:d4:23:54 | NULL | 1 | 4 | 4 | 2261 |
iqn.2005-03.org.open-iscsi:8191f9f922ef | NULL |
XenServer | 6.0.2 | 16190149248 |
com.cloud.hypervisor.xen.resource.XenServer602Resource | 4.2.0-SNAPSHOT | NULL
| NULL | xen-3.0-x86_64 ,
xen-3.0-x86_32p , hvm-3.0-x86_32 , hvm-3.0-x86_32p , hvm-3.0-x86_64 |
b127d031-d3bc-4859-bc52-633962ba61a9 | 1 |
1 | 0 | 1334635374 | NULL | 2013-04-19 00:20:22 |
2013-04-18 21:23:10 | NULL | 90 | Enabled | NULL | NULL
| Disabled |
| 9 | Rack3Host19.lab.vmops.com |
565dff5f-e17a-4216-a20a-6283e2bab0bf | Down | Routing |
10.223.59.4 | 255.255.255.192 | bc:30:5b:d4:15:d2 | 10.223.59.4
| 255.255.255.192 | bc:30:5b:d4:15:d2 | NULL | NULL
| NULL | 6 | 10.223.59.4 | 255.255.255.192 |
bc:30:5b:d4:15:d2 | NULL | 1 | 4 | 4 | 2261 |
iqn.2005-03.org.open-iscsi:f3aa69c5a08c | NULL |
XenServer | 6.0.2 | 3701658240 |
com.cloud.hypervisor.xen.resource.XenServer602Resource | 4.2.0-SNAPSHOT | NULL
| NULL | xen-3.0-x86_64 ,
xen-3.0-x86_32p , hvm-3.0-x86_32 , hvm-3.0-x86_32p , hvm-3.0-x86_64 |
bae94938-b22d-4874-8f18-1e9615338b3c | 1 |
1 | 0 | 1334637141 | NULL | 2013-04-19 00:20:22 |
2013-04-18 21:23:22 | NULL | 67 | Enabled
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
