manasaveloori created CLOUDSTACK-5848:
-----------------------------------------
Summary: "Unable to parse VLAN tag" message when network
associated with SRX external firewall device is rebooted.
Key: CLOUDSTACK-5848
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5848
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Network Devices
Affects Versions: 4.3.0
Environment: upgraded the CS from 3.0.6 patch E to 4.3
Reporter: manasaveloori
Priority: Critical
Fix For: 4.3.0
Steps:
1. Deploy CS 3.0.6 patch E using Xen 6.0.2 HV.
2. Add SRX device.
3 . Deploy the VMs uing the SRX network.
4. Enable PF,Static nat rules for the VM using SRX.
5. Upgrade the CS to 4.3.
6. Now restart the network associated with SRX or disable static nat etc.
Observing following exceptions in logs:
er root, class super-user --><rpc-reply
xmlns:junos="http://xml.juniper.net/junos/10.4R6/junos";><xnm:warning
xmlns="http://xml.juniper.net/xnm/1.1/xnm";
xmlns:xnm="http://xml.juniper.net/xnm/1.1/xnm";><message>uncommitted changes
will be discarded on exit</message></xnm:warning></rpc-reply>
2014-01-09 22:27:14,815 DEBUG [c.c.n.r.JuniperSrxResource]
(DirectAgent-22:ctx-357d339b) Opened a private configuration.
2014-01-09 22:27:14,816 ERROR [c.c.n.r.JuniperSrxResource]
(DirectAgent-22:ctx-357d339b) com.cloud.utils.exception.ExecutionException:
Unable to parse VLAN tag: 47
2014-01-09 22:27:14,816 DEBUG [c.c.n.r.JuniperSrxResource]
(DirectAgent-22:ctx-357d339b) Sending request: <!--Licensed to the Apache
Software Foundation (ASF) under oneor more contributor license agreements. See
the NOTICE filedistributed with this work for additional informationregarding
copyright ownership. The ASF licenses this fileto you under the Apache
License, Version 2.0 (the"License"); you may not use this file except in
compliancewith the License. You may obtain a copy of the License
athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law
or agreed to in writing,software distributed under the License is distributed
on an"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANYKIND, either express
or implied. See the License for thespecific language governing permissions and
limitationsunder the License.--><rpc><close-configuration/></rpc>
2014-01-09 22:27:14,868 DEBUG [c.c.n.r.JuniperSrxResource]
(DirectAgent-22:ctx-357d339b) Checking response: <rpc-reply
xmlns:junos="http://xml.juniper.net/junos/10.4R6/junos";></rpc-reply>
2014-01-09 22:27:14,868 DEBUG [c.c.n.r.JuniperSrxResource]
(DirectAgent-22:ctx-357d339b) Closed private configuration.
2014-01-09 22:27:14,869 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-22:ctx-357d339b) Seq 9-550895625: Response Received:
2014-01-09 22:27:14,869 DEBUG [c.c.a.t.Request] (DirectAgent-22:ctx-357d339b)
Seq 9-550895625: Processing: { Ans: , MgmtId: 6642334695485, via: 9, Ver: v1,
Flags: 10, [{"com.cloud.agent.api.Answer":{"result":false,"details":"Exception:
com.cloud.utils.exception.ExecutionException\nMessage: Unable to parse VLAN
tag: 47\nStack: com.cloud.utils.exception.ExecutionException: Unable to parse
VLAN tag: 47\n\tat
com.cloud.network.resource.JuniperSrxResource.getVlanTag(JuniperSrxResource.java:3609)\n\tat
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:894)\n\tat
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912)\n\tat
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912)\n\tat
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:830)\n\tat
com.cloud.network.resource.JuniperSrxResource.executeRequest(JuniperSrxResource.java:353)\n\tat
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)\n\tat
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)\n\tat
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)\n\tat
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)\n\tat
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)\n\tat
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)\n\tat
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)\n\tat
java.util.concurrent.FutureTask.run(FutureTask.java:166)\n\tat
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
java.lang.Thread.run(Thread.java:679)\n","wait":0}}] }
2014-01-09 22:27:14,870 DEBUG [c.c.a.t.Request] (Job-Executor-37:ctx-5ce0e0b1
ctx-50987553) Seq 9-550895625: Received: { Ans: , MgmtId: 6642334695485, via:
9, Ver: v1, Flags: 10, { Answer } }
2014-01-09 22:27:14,870 DEBUG [c.c.a.m.AgentManagerImpl]
(Job-Executor-37:ctx-5ce0e0b1 ctx-50987553) Details from executing class
com.cloud.agent.api.routing.SetFirewallRulesCommand: Exception:
com.cloud.utils.exception.ExecutionException
Message: Unable to parse VLAN tag: 47
Stack: com.cloud.utils.exception.ExecutionException: Unable to parse VLAN tag:
47
at
com.cloud.network.resource.JuniperSrxResource.getVlanTag(JuniperSrxResource.java:3609)
at
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:894)
at
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912)
at
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912)
at
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:830)
at
com.cloud.network.resource.JuniperSrxResource.executeRequest(JuniperSrxResource.java:353)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2014-01-09 22:27:14,870 ERROR [c.c.n.ExternalFirewallDeviceManagerImpl]
(Job-Executor-37:ctx-5ce0e0b1 ctx-50987553) External firewall was unable to
apply static nat rules to the SRX appliance in zone zonexen due to: Exception:
com.cloud.utils.exception.ExecutionException
Message: Unable to parse VLAN tag: 47
Stack: com.cloud.utils.exception.ExecutionException: Unable to parse VLAN tag:
47
at
com.cloud.network.resource.JuniperSrxResource.getVlanTag(JuniperSrxResource.java:3609)
at
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:894)
at
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912)
at
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912)
at
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:830)
at
com.cloud.network.resource.JuniperSrxResource.executeRequest(JuniperSrxResource.java:353)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
.
2014-01-09 22:27:14,871 WARN [c.c.n.f.FirewallManagerImpl]
(Job-Executor-37:ctx-5ce0e0b1 ctx-50987553) Failed to apply firewall rules due
to
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is
unreachable: External firewall was unable to apply static nat rules to the SRX
appliance in zone zonexen due to: Exception:
com.cloud.utils.exception.ExecutionException
Message: Unable to parse VLAN tag: 47
Stack: com.cloud.utils.exception.ExecutionException: Unable to parse VLAN tag:
47
at
com.cloud.network.resource.JuniperSrxResource.getVlanTag(JuniperSrxResource.java:3609)
at
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:894)
at
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912)
at
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:912)
at
com.cloud.network.resource.JuniperSrxResource.execute(JuniperSrxResource.java:830)
at
com.cloud.network.resource.JuniperSrxResource.executeRequest(JuniperSrxResource.java:353)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
.
at
com.cloud.network.ExternalFirewallDeviceManagerImpl.sendFirewallRules(ExternalFirewallDeviceManagerImpl.java:616)
at
com.cloud.network.ExternalFirewallDeviceManagerImpl.applyFirewallRules(ExternalFirewallDeviceManagerImpl.java:573)
at
com.cloud.network.element.JuniperSRXExternalFirewallElement.applyFWRules(JuniperSRXExternalFirewallElement.java:233)
at
com.cloud.network.firewall.FirewallManagerImpl.applyRules(FirewallManagerImpl.java:569)
at
com.cloud.network.IpAddressManagerImpl.applyRules(IpAddressManagerImpl.java:502)
at
com.cloud.network.firewall.FirewallManagerImpl.applyRules(FirewallManagerImpl.java:523)
at
com.cloud.network.firewall.FirewallManagerImpl.applyFirewallRules(FirewallManagerImpl.java:643)
at
com.cloud.network.firewall.FirewallManagerImpl.applyIngressFirewallRules(FirewallManagerImpl.java:616)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy174.applyIngressFirewallRules(Unknown Source)
at
org.apache.cloudstack.api.command.user.firewall.CreateFirewallRuleCmd.execute(CreateFirewallRuleCmd.java:125)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
at
com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
at
com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:524)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2014-01-09 22:27:14,964 DEBUG [c.c.a.t.Request] (Job-Executor-37:ctx-5ce0e0b1
ctx-50987553) Seq 9-550895626: Sending { Cmd , MgmtId: 6642334695485, via:
9(200-JuniperSRXFirewall-10.147.40.3), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.routing.SetFirewallRulesCommand":{"rules":[{"id":9,"srcVlanTag":"47","srcIp":"10.147.47.11","protocol":"tcp","srcPortRange":[22,22],"revoked":false,"alreadyAdded":true,"sourceCidrList":[],"purpose":"Firewall","defaultEgressPolicy":false},{"id":20,"srcVlanTag":"47","srcIp":"10.147.47.11","protocol":"tcp","srcPortRange":[1,655],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","defaultEgressPolicy":false},{"id":24,"srcVlanTag":"47","srcIp":"10.147.47.11","protocol":"tcp","srcPortRange":[1,65],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","defaultEgressPolicy":false}],"accessDetails":{},"wait":0}}]
}
2014-01-09 22:27:14,966 DEBUG [c.c.a.t.Request] (Job-Executor-37:ctx-5ce0e0b1
ctx-50987553) Seq 9-550895626: Executing: { Cmd , MgmtId: 6642334695485, via:
9(200-JuniperSRXFirewall-10.147.40.3), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.routing.SetFirewallRulesCommand":{"rules":[{"id":9,"srcVlanTag":"47","srcIp":"10.147.47.11","protocol":"tcp","srcPortRange":[22,22],"revoked":false,"alreadyAdded":true,"sourceCidrList":[],"purpose":"Firewall","defaultEgressPolicy":false},{"id":20,"srcVlanTag":"47","srcIp":"10.147.47.11","protocol":"tcp","srcPortRange":[1,655],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","defaultEgressPolicy":false},{"id":24,"srcVlanTag":"47","srcIp":"10.147.47.11","protocol":"tcp","srcPortRange":[1,65],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","defaultEgressPolicy":false}],"accessDetails":{},"wait":0}}]
}
2014-01-09 22:27:14,967 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-332:ctx-e5cfb854) Seq 9-550895626: Executing request
2014-01-09 22:27:14,996 DEBUG [c.c.n.r.JuniperSrxResource]
(DirectAgent-332:ctx-e5cfb854) Sending login request
2014-01-09 22:27:15,114 DEBUG [c.c.n.r.JuniperSrxResource]
(DirectAgent-332:ctx-e5cfb854) Checking login response
2014-01-09 22:27:15,114 DEBUG [c.c.n.r.JuniperSrxResource]
(DirectAgent-332:ctx-e5cfb854) Sending request: <!--Licensed to the Apache
Software Foundation (ASF) under oneor more contributor license agreements. See
the NOTICE filedistributed with this work for additional informationregarding
copyright ownership. The ASF licenses this fileto you under the Apache
License, Version 2.0 (the"License"); you may not use this file except in
compliancewith the License. You may obtain a copy of the License
athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law
or agreed to in writing,software distributed under the License is distributed
on an"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANYKIND, either express
or implied. See the License for thespecific language governing permissions and
limitationsunder the
License.--><rpc><open-configuration><private/></open-configuration></rpc>
2014-01-09 22:27:16,810 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-11:null) SeqA 8-162: Processing Seq 8-162: { Cmd ,
MgmtId: -1, via: 8, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":4,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2014-01-09 22:27:16,820 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-11:null) SeqA 8-162: Sending Seq 8-162: { Ans: , MgmtId:
6642334695485, via: 8, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2014-01-09 22:27:16,891 DEBUG [c.c.a.ApiServlet] (catalina-exec-3:ctx-b061c17c)
===START=== 10.252.192.34 -- GET command=queryAsyncJobResult&jobId=eac
attaching the MS logs
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
