Rayees Namathponnan created CLOUDSTACK-5047:
-----------------------------------------------
Summary: [Automation] KVM - Failed to add ACLlist to vpc network,
observed NPE in agent
Key: CLOUDSTACK-5047
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5047
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: KVM, Network Controller
Affects Versions: 4.3.0
Environment: KVM
Build : 4.3.0 master
Reporter: Rayees Namathponnan
Priority: Blocker
Fix For: 4.3.0
BVT test case
integration.smoke.test_privategw_acl.TestPrivateGwACL.test_privategw_acl
failed with master build
observed below error in MS log, while adding ACL list
2013-11-05 14:50:30,265 DEBUG [c.c.v.UserVmManagerImpl]
(AccountChecker-1:ctx-c25f1452) Firewall rules are removed successfully as a
part of vm id=6 expunge
2013-11-05 14:50:30,267 DEBUG [c.c.n.r.RulesManagerImpl]
(AccountChecker-1:ctx-c25f1452) No port forwarding rules are found for vm id=6
2013-11-05 14:50:30,267 DEBUG [c.c.v.UserVmManagerImpl]
(AccountChecker-1:ctx-c25f1452) Port forwarding rules are removed successfully
as a part of vm id=6 exp
unge
2013-11-05 14:50:30,271 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl]
(AccountChecker-1:ctx-c25f1452) Set load balancer rule for revoke: rule id 49,
vmId 6
2013-11-05 14:50:30,281 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl]
(AccountChecker-1:ctx-c25f1452) Set load balancer rule for revoke: rule id 49,
vmId 6
2013-11-05 14:50:30,288 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-0e021cd8) Found 2 routers to update status.
2013-11-05 14:50:30,289 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-0e021cd8) Found 0 networks to update RvR status.
2013-11-05 14:50:30,316 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(AccountChecker-1:ctx-c25f1452) Applying ip association in network
Ntwk[208|Guest|8]
2013-11-05 14:50:30,332 DEBUG [c.c.a.t.Request] (AccountChecker-1:ctx-c25f1452)
Seq 1-193922916: Sending { Cmd , MgmtId: 29066118877352, via: 1(Rack2Host11.la
b.vmops.com), Ver: v1, Flags: 100001,
[{"com.cloud.agent.api.routing.IpAssocCommand":{"ipAddresses":[{"accountId":6,"publicIp":"10.223.122.68","sourceNat":true
,"add":true,"oneToOneNat":false,"firstIP":true,"broadcastUri":"vlan://1221","vlanGateway":"10.223.122.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:9
6:6a:00:00:3d","networkRate":200,"trafficType":"Public"},{"accountId":6,"publicIp":"10.223.122.76","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":f
alse,"broadcastUri":"vlan://1221","vlanGateway":"10.223.122.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:c2:af:00:00:3d","networkRate":200,"trafficT
ype":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.3.181","router.name":"r-8-QA"},"wait":0}}]
}
2013-11-05 14:50:30,627 DEBUG [c.c.a.t.Request] (AgentManager-Handler-11:null)
Seq 1-193922916: Processing: { Ans: , MgmtId: 29066118877352, via: 1, Ver: v1,
Flags: 0,
[{"com.cloud.agent.api.Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat
com.cloud.agent.api.routing.IpAssocAnswer.<init>(IpAss
ocAnswer.java:34)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2062)\n\tat
com.cloud.hypervisor.kvm.r
esource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1278)\n\tat
com.cloud.agent.Agent.processRequest(Agent.java:498)\n\tat com.cloud.
agent.Agent$AgentRequestHandler.doTask(Agent.java:806)\n\tat
com.cloud.utils.nio.Task.run(Task.java:83)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1110)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
java.lang.Thread.run(Thread.java:679
)\n","wait":0}}] }
2013-11-05 14:50:30,627 DEBUG [c.c.a.t.Request] (AccountChecker-1:ctx-c25f1452)
Seq 1-193922916: Received: { Ans: , MgmtId: 29066118877352, via: 1, Ver: v1, F
lags: 0, { Answer } }
2013-11-05 14:50:30,627 WARN [c.c.n.l.LoadBalancingRulesManagerImpl]
(AccountChecker-1:ctx-c25f1452) Unable to apply the load balancer config
because resource
is unavaliable.
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is
unreachable: Unable to apply ip association on router
at
com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3797)
at
com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:3593)
at
com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:505)
at sun.reflect.GeneratedMethodAccessor391.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
:
Observed below NPE in KVM agent
/primary -m /mnt/fff90cb5-06dd-33b3-8815-d78c08ca01d9 -h 10.223.50.66
2013-11-05 14:51:30,323 DEBUG [kvm.resource.KVMHAMonitor] (Thread-315:null)
Execution is successful.
2013-11-05 14:51:30,325 DEBUG [kvm.resource.KVMHAMonitor] (Thread-315:null)
Found NFS storage pool 41b632b5-40b3-3024-a38b-ea259c72579f in libvirt,
continuing
2013-11-05 14:51:30,325 DEBUG [kvm.resource.KVMHAMonitor] (Thread-315:null)
Executing:
/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
10.223.110.232 -p /export/home/rayees/SC_QA_AUTO4/primary2 -m
/mnt/41b632b5-40b3-3024-a38b-ea259c72579f -h 10.223.50.66
2013-11-05 14:51:30,344 DEBUG [kvm.resource.KVMHAMonitor] (Thread-315:null)
Execution is successful.
2013-11-05 14:51:30,391 DEBUG [kvm.resource.BridgeVifDriver]
(agentRequest-Handler-4:null) Execution is successful.
2013-11-05 14:51:30,391 DEBUG [kvm.resource.BridgeVifDriver]
(agentRequest-Handler-4:null) Set name-type for VLAN subsystem. Should be
visible in /proc/net/vlan/config
2013-11-05 14:51:30,522 WARN [cloud.agent.Agent] (agentRequest-Handler-4:null)
Caught:
java.lang.NullPointerException
at
com.cloud.agent.api.routing.IpAssocAnswer.<init>(IpAssocAnswer.java:34)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2062)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1278)
at com.cloud.agent.Agent.processRequest(Agent.java:498)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:806)
at com.cloud.utils.nio.Task.run(Task.java:83)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-11-05 14:51:30,522 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null)
Seq 1-193922921: { Ans: , MgmtId: 29066118877352, via: 1, Ver: v1, Flags: 0,
[{"com.cloud.agent.api.Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat
com.cloud.agent.api.routing.IpAssocAnswer.<init>(IpAssocAnswer.java:34)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2062)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1278)\n\tat
com.cloud.agent.Agent.processRequest(Agent.java:498)\n\tat
com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:806)\n\tat
com.cloud.utils.nio.Task.run(Task.java:83)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
java.lang.Thread.run(Thread.java:679)\n","wait":0}}] }
2013-11-05 14:51:35,077 DEBUG [kvm.resource.LibvirtComputingResource]
(UgentTask-5:null) Executing:
/usr/share/cloudstack-common/scripts/vm/network/security_group.py
get_rule_logs_for_vms
2013-11-05 14:51:35,621 DEBUG [kvm.resource.LibvirtComputingResource]
(UgentTask-5:null) Execution is successful.
2013-11-05 14:51:35,621 DEBUG [cloud.agent.Agent] (UgentTask-5:null) Sending
ping: Seq 1-308: { Cmd , MgmtId: -1, via: 1, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"newStates":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":1,"wait":0}}]
}
2013-11-05 14:51:35,627 DEBUG [cloud.agent.Agent] (Agent-Handler-4:null)
Received response: Seq 1-308: { Ans: , MgmtId: 29066118877352, via: 1, Ver:
v1, Flags: 100010,
[{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":1,"wait":0},"result":true,"wait":0}}]
}
(END)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
