Wei Zhou created CLOUDSTACK-9055: ------------------------------------ Summary: NPE in update Redundant State of VPC networks Key: CLOUDSTACK-9055 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9055 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Reporter: Wei Zhou Assignee: Wei Zhou
2015-11-11 08:32:48,695 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-ee477aaa) Found 4 routers to update status. 2015-11-11 08:32:48,697 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-ee477aaa) Found 1 VPC networks to update Redundant State. 2015-11-11 08:32:48,697 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-ee477aaa) Found 0 networks to update RvR status. 2015-11-11 08:32:48,705 DEBUG [c.c.a.t.Request] (RedundantRouterStatusMonitor-7:ctx-195132c6) Seq 22-1391893759834266981: Sending { Cmd , MgmtId: 345051313197, via: 22(node12), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.CheckRouterCommand":{"accessDetails":{"router.name":"r-7548-VM","router.ip":"169.254.2.52"},"wait":30}}] } 2015-11-11 08:32:58,451 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-f907d806) Begin cleanup expired async-jobs 2015-11-11 08:32:58,455 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-f907d806) End cleanup expired async-jobs 2015-11-11 08:33:06,370 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-ab0d6092) Zone 1 is ready to launch console proxy 2015-11-11 08:33:06,435 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-2b4d78f6) Zone 1 is ready to launch secondary storage VM 2015-11-11 08:33:08,451 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-a8a81c83) Begin cleanup expired async-jobs 2015-11-11 08:33:08,455 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-a8a81c83) End cleanup expired async-jobs 2015-11-11 08:33:15,441 DEBUG [c.c.s.StatsCollector] (StatsCollector-6:ctx-bba99e59) HostStatsCollector is running... 2015-11-11 08:33:17,649 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-c64bf7b6) AutoScaling Monitor is running... 2015-11-11 08:33:18,451 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-73c3e11b) Begin cleanup expired async-jobs 2015-11-11 08:33:18,455 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-73c3e11b) End cleanup expired async-jobs 2015-11-11 08:33:18,660 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-25d2c01c) Found 4 routers to update status. 2015-11-11 08:33:18,663 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-25d2c01c) Found 1 VPC networks to update Redundant State. 2015-11-11 08:33:18,664 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-25d2c01c) Found 0 networks to update RvR status. 2015-11-11 08:33:18,673 DEBUG [c.c.a.t.Request] (RedundantRouterStatusMonitor-5:ctx-db2e782d) Seq 22-1391893759834266982: Sending { Cmd , MgmtId: 345051313197, via: 22(node12), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.CheckRouterCommand":{"accessDetails":{"router.name":"r-7548-VM","router.ip":"169.254.2.52"},"wait":30}}] } 2015-11-11 08:33:18,674 DEBUG [c.c.a.m.AgentAttache] (RedundantRouterStatusMonitor-10:ctx-7afb35e1) Seq 22-1391893759834266980: Waiting some more time because this is the current command 2015-11-11 08:33:18,695 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-8466fef5) Found 4 routers to update status. 2015-11-11 08:33:18,697 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-8466fef5) Found 1 VPC networks to update Redundant State. /RedundantRouterStatusMonitor-10 2015-11-11 08:33:18,698 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-8466fef5) Found 0 networks to update RvR status. 2015-11-11 08:33:18,705 DEBUG [c.c.a.t.Request] (RedundantRouterStatusMonitor-6:ctx-b997744e) Seq 22-1391893759834266983: Sending { Cmd , MgmtId: 345051313197, via: 22(node12), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.CheckRouterCommand":{"accessDetails":{"router.name":"r-7548-VM","router.ip":"169.254.2.52"},"wait":30}}] } 2015-11-11 08:33:18,706 DEBUG [c.c.a.m.AgentAttache] (RedundantRouterStatusMonitor-7:ctx-195132c6) Seq 22-1391893759834266981: Waiting some more time because this is the current command 2015-11-11 08:33:28,451 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-d9616dea) Begin cleanup expired async-jobs 2015-11-11 08:33:28,455 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-d9616dea) End cleanup expired async-jobs 2015-11-11 08:33:36,371 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-6034ab55) Zone 1 is ready to launch console proxy 2015-11-11 08:33:36,435 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-06c87dbe) Zone 1 is ready to launch secondary storage VM 2015-11-11 08:33:38,451 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-5564f450) Begin cleanup expired async-jobs 2015-11-11 08:33:38,455 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-5564f450) End cleanup expired async-jobs 2015-11-11 08:33:48,451 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-0729ed82) Begin cleanup expired async-jobs 2015-11-11 08:33:48,456 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-0729ed82) End cleanup expired async-jobs 2015-11-11 08:33:48,660 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-a256ed6c) Found 4 routers to update status. 2015-11-11 08:33:48,662 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-a256ed6c) Found 1 VPC networks to update Redundant State. 2015-11-11 08:33:48,664 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-a256ed6c) Found 0 networks to update RvR status. 2015-11-11 08:33:48,676 DEBUG [c.c.a.m.AgentAttache] (RedundantRouterStatusMonitor-10:ctx-7afb35e1) Seq 22-1391893759834266980: Waiting some more time because this is the current command 2015-11-11 08:33:48,676 DEBUG [c.c.a.m.AgentAttache] (RedundantRouterStatusMonitor-5:ctx-db2e782d) Seq 22-1391893759834266982: Waiting some more time because this is the current command 2015-11-11 08:33:48,677 INFO [c.c.u.e.CSExceptionErrorCode] (RedundantRouterStatusMonitor-10:ctx-7afb35e1) Could not find exception: com.cloud.exception.OperationTimedoutException in error code list for exceptions 2015-11-11 08:33:48,677 WARN [c.c.a.m.AgentAttache] (RedundantRouterStatusMonitor-10:ctx-7afb35e1) Seq 22-1391893759834266980: Timed out on Seq 22-1391893759834266980: { Cmd , MgmtId: 345051313197, via: 22(node12), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.CheckRouterCommand":{"accessDetails":{"router.name":"r-7548-VM","router.ip":"169.254.2.52"},"wait":30}}] } 2015-11-11 08:33:48,677 DEBUG [c.c.a.m.AgentAttache] (RedundantRouterStatusMonitor-10:ctx-7afb35e1) Seq 22-1391893759834266980: Cancelling. 2015-11-11 08:33:48,677 WARN [c.c.a.m.AgentManagerImpl] (RedundantRouterStatusMonitor-10:ctx-7afb35e1) Operation timed out: Commands 1391893759834266980 to Host 22 timed out after 60 2015-11-11 08:33:48,678 WARN [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RedundantRouterStatusMonitor-10:ctx-7afb35e1) Unable to update router r-7548-VM's status 2015-11-11 08:33:48,678 ERROR [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RedundantRouterStatusMonitor-10:ctx-7afb35e1) Fail to complete the RvRStatusUpdateTask! java.lang.NullPointerException at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.updateRoutersRedundantState(VirtualNetworkApplianceManagerImpl.java:1019) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl$RvRStatusUpdateTask.runInContext(VirtualNetworkApplianceManagerImpl.java:1201) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2015-11-11 08:33:48,683 DEBUG [c.c.a.t.Request] (RedundantRouterStatusMonitor-6:ctx-d0632264) Seq 22-1391893759834266984: Sending { Cmd , MgmtId: 345051313197, via: 22(node12), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.CheckRouterCommand":{"accessDetails":{"router.name":"r-7548-VM","router.ip":"169.254.2.52"},"wait":30}}] } -- This message was sent by Atlassian JIRA (v6.3.4#6332)