[
https://issues.apache.org/jira/browse/CLOUDSTACK-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
manasaveloori reopened CLOUDSTACK-5325:
---------------------------------------
Network restart is failing with KVM HV..But it is not failing using Xen HV.
The exceptions seen in log for KVM HV in MS logs:
2013-12-19 02:30:03,640 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Restarting network 215...
2013-12-19 02:30:03,640 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Skip the shutting down of network
id=215
2013-12-19 02:30:03,640 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Implementing the network
Ntwk[215|Guest|8] elements and resources as a part of network restart
2013-12-19 02:30:03,646 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Asking VirtualRouter to implemenet
Ntwk[215|Guest|8]
2013-12-19 02:30:03,649 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Lock is acquired for network id 215
as a part of router startup in
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(3)-Pod(null)-Cluster(null)-Host(null)-Storage()]
2013-12-19 02:30:03,652 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Lock is released for network id 215
as a part of router startup in
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(3)-Pod(null)-Cluster(null)-Host(null)-Storage()]
2013-12-19 02:30:03,652 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Reprogramming network
Ntwk[215|Guest|8] as a part of network implement
2013-12-19 02:30:03,655 DEBUG [c.c.n.r.RulesManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) There are no static nat to apply
for network id=215
2013-12-19 02:30:03,674 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Applying firewall rules in network
Ntwk[215|Guest|8]
2013-12-19 02:30:03,691 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f
ctx-3eeba0e9) Seq 5-2014318288: Sending { Cmd , MgmtId: 233845177509765, via:
5(Rack1Pod1Host28), Ver: v1, Flags: 100001,
[{"com.cloud.agent.api.routing.SetFirewallRulesCommand":{"rules":[{"id":11,"srcIp":"10.147.47.36","protocol":"udp","srcPortRange":[500,500],"revoked":false,"alreadyAdded":true,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Ingress","defaultEgressPolicy":false},{"id":13,"srcIp":"10.147.47.36","protocol":"udp","srcPortRange":[1701,1701],"revoked":false,"alreadyAdded":true,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Ingress","defaultEgressPolicy":false},{"id":15,"srcIp":"10.147.47.36","protocol":"udp","srcPortRange":[4500,4500],"revoked":false,"alreadyAdded":true,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Ingress","defaultEgressPolicy":false},{"id":16,"srcIp":"10.147.47.36","protocol":"tcp","srcPortRange":[1,65535],"revoked":false,"alreadyAdded":true,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Ingress","defaultEgressPolicy":false},{"id":26,"srcIp":"10.147.47.34","protocol":"tcp","srcPortRange":[1,65535],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Ingress","defaultEgressPolicy":false},{"id":29,"srcIp":"10.147.47.31","protocol":"tcp","srcPortRange":[1,65535],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Ingress","defaultEgressPolicy":false}],"accessDetails":{"router.guest.ip":"10.1.1.1","firewall.egress.default":"false","zone.network.type":"Advanced","router.ip":"169.254.2.217","router.name":"r-39-VM"},"wait":0}}]
}
2013-12-19 02:30:03,946 DEBUG [c.c.a.t.Request] (StatsCollector-2:ctx-b2fc8c8c)
Seq 5-2014318287: Received: { Ans: , MgmtId: 233845177509765, via: 5, Ver: v1,
Flags: 10, { GetHostStatsAnswer } }
2013-12-19 02:30:03,982 DEBUG [c.c.a.t.Request] (AgentManager-Handler-4:null)
Seq 5-2014318288: Processing: { Ans: , MgmtId: 233845177509765, via: 5, Ver:
v1, Flags: 0,
[{"com.cloud.agent.api.routing.SetFirewallRulesAnswer":{"result":true,"wait":0}}]
}
2013-12-19 02:30:03,982 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f
ctx-3eeba0e9) Seq 5-2014318288: Received: { Ans: , MgmtId: 233845177509765,
via: 5, Ver: v1, Flags: 0, { SetFirewallRulesAnswer } }
2013-12-19 02:30:04,001 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Applying ip association in network
Ntwk[215|Guest|8]
2013-12-19 02:30:04,020 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f
ctx-3eeba0e9) Seq 5-2014318289: Sending { Cmd , MgmtId: 233845177509765, via:
5(Rack1Pod1Host28), Ver: v1, Flags: 100001,
[{"com.cloud.agent.api.routing.IpAssocCommand":{"ipAddresses":[{"accountId":3,"publicIp":"10.147.47.36","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"broadcastUri":"47","vlanGateway":"10.147.47.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:e3:00:00:00:17","networkRate":200,"trafficType":"Public"},{"accountId":3,"publicIp":"10.147.47.31","sourceNat":false,"add":false,"oneToOneNat":false,"firstIP":false,"broadcastUri":"47","vlanGateway":"10.147.47.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:5f:8d:00:00:17","networkRate":200,"trafficType":"Public"},{"accountId":3,"publicIp":"10.147.47.34","sourceNat":false,"add":false,"oneToOneNat":false,"firstIP":false,"broadcastUri":"47","vlanGateway":"10.147.47.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:5f:8d:00:00:17","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.217","router.name":"r-39-VM"},"wait":0}}]
}
2013-12-19 02:30:04,839 DEBUG [c.c.a.t.Request] (AgentManager-Handler-14:null)
Seq 5-2014318289: Processing: { Ans: , MgmtId: 233845177509765, via: 5, Ver:
v1, Flags: 0,
[{"com.cloud.agent.api.routing.IpAssocAnswer":{"results":["Failed","Failed","Failed"],"result":false,"wait":0}}]
}
2013-12-19 02:30:04,839 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f
ctx-3eeba0e9) Seq 5-2014318289: Received: { Ans: , MgmtId: 233845177509765,
via: 5, Ver: v1, Flags: 0, { IpAssocAnswer } }
2013-12-19 02:30:04,839 WARN [c.c.n.f.FirewallManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Failed to apply firewall rules due
to
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:3] is
unreachable: Unable to apply ip association on router
at
com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3735)
at
com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:3526)
at
com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:505)
at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy240.associatePublicIP(Unknown Source)
at
com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:476)
at
com.cloud.network.IpAddressManagerImpl.applyIpAssociations(IpAddressManagerImpl.java:976)
at
com.cloud.network.IpAddressManagerImpl.applyRules(IpAddressManagerImpl.java:514)
at
com.cloud.network.firewall.FirewallManagerImpl.applyRules(FirewallManagerImpl.java:521)
at
com.cloud.network.firewall.FirewallManagerImpl.applyFirewallRules(FirewallManagerImpl.java:641)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy175.applyFirewallRules(Unknown Source)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.reprogramNetworkRules(NetworkOrchestrator.java:1090)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1063)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.restartNetwork(NetworkOrchestrator.java:2370)
at
com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1869)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy199.restartNetwork(Unknown Source)
at
org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
at
com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
at
com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:521)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-12-19 02:30:04,841 WARN [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Failed to reapply Ingress firewall
rule(s) as a part of network id=215 restart
2013-12-19 02:30:04,844 DEBUG [c.c.n.f.FirewallManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) applying default firewall egress
rules
2013-12-19 02:30:04,860 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Applying firewall rules in network
Ntwk[215|Guest|8]
2013-12-19 02:30:04,867 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f
ctx-3eeba0e9) Seq 5-2014318290: Sending { Cmd , MgmtId: 233845177509765, via:
5(Rack1Pod1Host28), Ver: v1, Flags: 100001,
[{"com.cloud.agent.api.routing.SetFirewallRulesCommand":{"rules":[{"id":31,"srcIp":"","protocol":"tcp","srcPortRange":[1,65535],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Egress","defaultEgressPolicy":false}],"accessDetails":{"router.guest.ip":"10.1.1.1","firewall.egress.default":"false","zone.network.type":"Advanced","router.ip":"169.254.2.217","router.name":"r-39-VM"},"wait":0}}]
}
2013-12-19 02:30:05,003 DEBUG [c.c.a.t.Request] (AgentManager-Handler-13:null)
Seq 5-2014318290: Processing: { Ans: , MgmtId: 233845177509765, via: 5, Ver:
v1, Flags: 0,
[{"com.cloud.agent.api.routing.SetFirewallRulesAnswer":{"results":["Failed"],"result":false,"wait":0}}]
}
2013-12-19 02:30:05,004 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f
ctx-3eeba0e9) Seq 5-2014318290: Received: { Ans: , MgmtId: 233845177509765,
via: 5, Ver: v1, Flags: 0, { SetFirewallRulesAnswer } }
2013-12-19 02:30:05,004 WARN [c.c.n.f.FirewallManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Failed to apply firewall rules due
to
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:3] is
unreachable: Unable to apply firewall rules on router
at
com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3735)
at
com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyFirewallRules(VirtualNetworkApplianceManagerImpl.java:3542)
at sun.reflect.GeneratedMethodAccessor353.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy240.applyFirewallRules(Unknown Source)
at
com.cloud.network.element.VirtualRouterElement.applyFWRules(VirtualRouterElement.java:248)
at
com.cloud.network.firewall.FirewallManagerImpl.applyRules(FirewallManagerImpl.java:567)
at
com.cloud.network.IpAddressManagerImpl.applyRules(IpAddressManagerImpl.java:502)
at
com.cloud.network.firewall.FirewallManagerImpl.applyRules(FirewallManagerImpl.java:521)
at
com.cloud.network.firewall.FirewallManagerImpl.applyFirewallRules(FirewallManagerImpl.java:641)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy175.applyFirewallRules(Unknown Source)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.reprogramNetworkRules(NetworkOrchestrator.java:1106)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1063)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.restartNetwork(NetworkOrchestrator.java:2370)
at
com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1869)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy199.restartNetwork(Unknown Source)
at
org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
at
com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
at
com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:521)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-12-19 02:30:05,006 WARN [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Failed to reapply firewall Egress
rule(s) as a part of network id=215 restart
2013-12-19 02:30:05,007 DEBUG [c.c.n.r.RulesManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) There are no port forwarding rules
to apply for network id=215
2013-12-19 02:30:05,009 DEBUG [c.c.n.r.RulesManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) There are no static nat rules to
apply for network id=215
2013-12-19 02:30:05,010 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Applying load balancer rules of
scheme Public in network id=215
2013-12-19 02:30:05,010 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) There are no Load Balancing Rules
to forward to the network elements
2013-12-19 02:30:05,011 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Applying load balancer rules of
scheme Internal in network id=215
2013-12-19 02:30:05,011 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) There are no Load Balancing Rules
to forward to the network elements
2013-12-19 02:30:05,028 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f
ctx-3eeba0e9) Seq 5-2014318291: Sending { Cmd , MgmtId: 233845177509765, via:
5(Rack1Pod1Host28), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.routing.VpnUsersCfgCommand":{"userpwds":[{"username":"root","add":true},{"username":"test","add":true}],"accessDetails":{"account.id":"3","zone.network.type":"Advanced","router.ip":"169.254.2.217","router.name":"r-39-VM"},"wait":0}},{"com.cloud.agent.api.routing.RemoteAccessVpnCfgCommand":{"create":true,"vpcEnabled":false,"vpnServerIp":"10.147.47.36","ipRange":"10.1.2.2-10.1.2.8","presharedKey":"hhtetEww7RvBOMHRXtx5k5jz","localIp":"10.1.2.1","localCidr":"10.1.1.0/24","publicInterface":"eth2","accessDetails":{"zone.network.type":"Advanced","router.ip":"169.254.2.217","router.name":"r-39-VM"},"wait":0}}]
}
2013-12-19 02:30:05,336 DEBUG [c.c.a.t.Request] (AgentManager-Handler-1:null)
Seq 5-2014318291: Processing: { Ans: , MgmtId: 233845177509765, via: 5, Ver:
v1, Flags: 110,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Configure
VPN failed","wait":0}}] }
2013-12-19 02:30:05,337 DEBUG [c.c.a.m.AgentAttache]
(AgentManager-Handler-1:null) Seq 5-2014318291: No more commands found
2013-12-19 02:30:05,337 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f
ctx-3eeba0e9) Seq 5-2014318291: Received: { Ans: , MgmtId: 233845177509765,
via: 5, Ver: v1, Flags: 110, { Answer, Answer } }
2013-12-19 02:30:05,337 ERROR [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Unable to start vpn in zone 3 for
account 3 on domR: r-39-VM due to Configure VPN failed
2013-12-19 02:30:05,337 WARN [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Failed to implement network
Ntwk[215|Guest|8] elements and resources as a part of network restart due to
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:3] is
unreachable: Unable to start vpn in zone 3 for account 3 on domR: r-39-VM due
to Configure VPN failed
at
com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRemoteAccessVpn(VirtualNetworkApplianceManagerImpl.java:2624)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy240.startRemoteAccessVpn(Unknown Source)
at
com.cloud.network.element.VirtualRouterElement.startVpn(VirtualRouterElement.java:430)
at
com.cloud.network.vpn.RemoteAccessVpnManagerImpl.startRemoteAccessVpn(RemoteAccessVpnManagerImpl.java:442)
at
com.cloud.network.vpn.RemoteAccessVpnManagerImpl.startRemoteAccessVpn(RemoteAccessVpnManagerImpl.java:92)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.reprogramNetworkRules(NetworkOrchestrator.java:1140)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1063)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.restartNetwork(NetworkOrchestrator.java:2370)
at
com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1869)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy199.restartNetwork(Unknown Source)
at
org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
at
com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
at
com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:521)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-12-19 02:30:05,339 WARN [c.c.n.NetworkServiceImpl]
(Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Network id=215 failed to restart.
2013-12-19 02:30:05,412 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Job-Executor-55:ctx-4f7c042f) Complete async job-118, jobStatus: FAILED,
resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed
to restart network"}
The same should happen for Xen.
> Restart Network command should be blocked on non upgraded router network
> ------------------------------------------------------------------------
>
> Key: CLOUDSTACK-5325
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5325
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Upgrade
> Affects Versions: 4.3.0
> Reporter: shweta agarwal
> Assignee: Alena Prokharchyk
> Priority: Critical
> Fix For: 4.3.0
>
>
> Restart Network command should be blocked on non upgraded router network due
> to following reasons:
> 1. It fails when there are rules like PF /LB /firewall to be applied as a
> part of network restart . But don't give proper error message just says
> restart networks failed .
> 2. If user tries restart network with force cleanup =true ; his network will
> be restarted with new router . In some sense that account owner has bumped
> up his priority w.r.t time on router upgrade
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
