[
https://issues.apache.org/jira/browse/CLOUDSTACK-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13723643#comment-13723643
]
Sowmya Krishnan edited comment on CLOUDSTACK-3938 at 7/30/13 9:47 AM:
----------------------------------------------------------------------
Here's the remaining part of the exceptions:
2013-07-30 02:34:02,492 INFO [utils.exception.CSExceptionErrorCode]
(StatsCollector-2:null) Could not find exception:
com.cloud.exception.OperationTimedoutE
xception in error code list for exceptions
2013-07-30 02:34:02,492 WARN [agent.manager.AgentAttache]
(StatsCollector-2:null) Seq 4-1100414979: Timed out on null
2013-07-30 02:34:02,492 DEBUG [agent.manager.AgentAttache]
(StatsCollector-2:null) Seq 4-1100414979: Cancelling.
2013-07-30 02:34:02,492 DEBUG [cloud.storage.StorageManagerImpl]
(StatsCollector-2:null) Unable to send storage pool command to
Pool[2|NetworkFilesystem] via
4
com.cloud.exception.OperationTimedoutException: Commands 1100414979 to Host 4
timed out after 3600
at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:430)
at
com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:486)
at
com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:439)
at
com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:977)
at
com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:428)
at
com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:442)
at
com.cloud.server.StatsCollector$StorageCollector.run(StatsCollector.java:562)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2013-07-30 02:34:02,493 INFO [cloud.server.StatsCollector]
(StatsCollector-2:null) Unable to reach Pool[2|NetworkFilesystem]
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:2] is
unreachable: Unable to send command to the pool
at
com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:987)
at
com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:428)
at
com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:442)
at
com.cloud.server.StatsCollector$StorageCollector.run(StatsCollector.java:562)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2013-07-30 02:34:02,500 DEBUG [agent.manager.ClusteredAgentAttache]
(StatsCollector-3:null) Seq 3-2136014854: Forwarding null to 206915885094132
2013-07-30 02:34:02,502 DEBUG [agent.manager.ClusteredAgentAttache]
(AgentManager-Handler-19:null) Seq 3-2136014854: Routing from 206915885097283
2013-07-30 02:34:02,503 DEBUG [agent.manager.ClusteredAgentAttache]
(AgentManager-Handler-19:null) Seq 3-2136014854: Link is closed
was (Author: sowmyak):
Here's the remaining part of the exceptions:
> Operation Timed out and Resource unreachable exceptions in clustered
> management server setup
> --------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-3938
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3938
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Management Server
> Affects Versions: 4.2.0
> Environment: 4.2, Using simulator to setup a load test env
> Reporter: Sowmya Krishnan
> Priority: Blocker
> Fix For: 4.2.0
>
>
> Set up:
> 2 Management servers, and mysql DB running in a remote server
> Using simulated hosts and resources.
> Deployed advanced zone with RVR, and setting up simulator hosts and storage
> pools.
> After deploying few of them, getting the following exceptions constantly:
> 2013-07-30 02:34:02,468 DEBUG [agent.transport.Request]
> (StatsCollector-2:null) Seq 2-197787660: Received: { Ans: , MgmtId:
> 206915885097283, via: 2, Ver: v1
> , Flags: 10, { GetStorageStatsAnswer } }
> 2013-07-30 02:34:02,472 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-35:null) Seq 1-1644953610: Response Received:
> 2013-07-30 02:34:02,472 DEBUG [agent.transport.Request]
> (StatsCollector-3:null) Seq 1-1644953610: Received: { Ans: , MgmtId:
> 206915885097283, via: 1, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2013-07-30 02:34:02,476 DEBUG [agent.manager.ClusteredAgentAttache]
> (StatsCollector-2:null) Seq 3-2136014853: Forwarding null to 206915885094132
> 2013-07-30 02:34:02,478 DEBUG [agent.manager.ClusteredAgentAttache]
> (AgentManager-Handler-15:null) Seq 3-2136014853: Routing from 206915885097283
> 2013-07-30 02:34:02,478 DEBUG [agent.manager.ClusteredAgentAttache]
> (AgentManager-Handler-15:null) Seq 3-2136014853: Link is closed
> 2013-07-30 02:34:02,478 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (AgentManager-Handler-15:null) Seq 3-2136014853: MgmtId 206915885097283: Req:
> Resource [Host:3] is unreachable: Host 3: Link is closed
> 2013-07-30 02:34:02,479 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (AgentManager-Handler-15:null) Seq 3--1: MgmtId 206915885097283: Req: Routing
> to peer
> 2013-07-30 02:34:02,481 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (AgentManager-Handler-16:null) Seq 3--1: MgmtId 206915885097283: Req: Cancel
> request received
> 2013-07-30 02:34:02,481 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-16:null) Seq 3-2136014853: Cancelling.
> 2013-07-30 02:34:02,481 DEBUG [agent.manager.AgentAttache]
> (StatsCollector-2:null) Seq 3-2136014853: Waiting some more time because this
> is the current command
> 2013-07-30 02:34:02,481 DEBUG [agent.manager.AgentAttache]
> (StatsCollector-2:null) Seq 3-2136014853: Waiting some more time because this
> is the current command
> 2013-07-30 02:34:02,481 INFO [utils.exception.CSExceptionErrorCode]
> (StatsCollector-2:null) Could not find exception:
> com.cloud.exception.OperationTimedoutException in error code list for
> exceptions
> 2013-07-30 02:34:02,482 WARN [agent.manager.AgentAttache]
> (StatsCollector-2:null) Seq 3-2136014853: Timed out on null
> 2013-07-30 02:34:02,482 DEBUG [agent.manager.AgentAttache]
> (StatsCollector-2:null) Seq 3-2136014853: Cancelling.
> 2013-07-30 02:34:02,482 DEBUG [cloud.storage.StorageManagerImpl]
> (StatsCollector-2:null) Unable to send storage pool command to
> Pool[2|NetworkFilesystem] via 3
> com.cloud.exception.OperationTimedoutException: Commands 2136014853 to Host 3
> timed out after 3600
> at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:430)
> at
> com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:486)
> at
> com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:439)
> at
> com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:977)
> at
> com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:428)
> at
> com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:442)
> at
> com.cloud.server.StatsCollector$StorageCollector.run(StatsCollector.java:562)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-30 02:34:02,483 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-34:null) Seq 2-197787661: Executing request
> 2013-07-30 02:34:02,488 DEBUG [agent.manager.ClusteredAgentAttache]
> (StatsCollector-2:null) Seq 4-1100414979: Forwarding null to 206915885094132
> 2013-07-30 02:34:02,489 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-34:null) Seq 2-197787661: Response Received:
> 2013-07-30 02:34:02,489 DEBUG [agent.transport.Request]
> (StatsCollector-3:null) Seq 2-197787661: Received: { Ans: , MgmtId:
> 206915885097283, via: 2, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2013-07-30 02:34:02,490 DEBUG [agent.manager.ClusteredAgentAttache]
> (AgentManager-Handler-17:null) Seq 4-1100414979: Routing from 206915885097283
> 2013-07-30 02:34:02,490 DEBUG [agent.manager.ClusteredAgentAttache]
> (AgentManager-Handler-17:null) Seq 4-1100414979: Link is closed
> 2013-07-30 02:34:02,490 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (AgentManager-Handler-17:null) Seq 4-1100414979: MgmtId 206915885097283: Req:
> Resource [Host:4] is unreachable: Host 4: Link is closed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
