Chandan Purushothama created CLOUDSTACK-2399: ------------------------------------------------
Summary: NTier: Deletion of Private Gateway among multiple private Gateways of the the VPC deletes the Wrong private gateway Key: CLOUDSTACK-2399 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2399 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Management Server Affects Versions: 4.2.0 Reporter: Chandan Purushothama Priority: Critical Fix For: 4.2.0 ================ Steps to Reproduce: ================ 1. Create a VPC. 2. Create Two private gateways to the VPC where two NICs to the private gateway network get created on the VPC VR. 3. Delete the first private Gateway created to the VPC. ========== Observations: ========== 2013-05-08 15:35:18,186 DEBUG [cloud.api.ApiServlet] (catalina-exec-1:null) ===START=== 10.216.132.104 -- GET command=deletePrivateGateway&id=b6fe4574-fe8f-4846-844d-9182e3217ceb&response=json&sessionkey=J5RF%2BKuEucFSQi3XIQmJhxorqOM%3D&_=1368052554491 2013-05-08 15:35:18,208 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-1:null) submit async job-66, details: AsyncJobVO {id:66, userId: 2, accountId: 2, sessionKey: null, instanceType: PrivateGateway, instanceId: null, cmd: org.apache.cloudstack.api.command.admin.vpc.DeletePrivateGatewayCmd, cmdOriginator: null, cmdInfo: {"response":"json","id":"b6fe4574-fe8f-4846-844d-9182e3217ceb","sessionkey":"J5RF+KuEucFSQi3XIQmJhxorqOM\u003d","ctxUserId":"2","httpmethod":"GET","_":"1368052554491","ctxAccountId":"2","ctxStartEventId":"183"}, cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, processStatus: 0, resultCode: 0, result: null, initMsid: 7508777239729, completeMsid: null, lastUpdated: null, lastPolled: null, created: null} 2013-05-08 15:35:18,209 DEBUG [cloud.api.ApiServlet] (catalina-exec-1:null) ===END=== 10.216.132.104 -- GET command=deletePrivateGateway&id=b6fe4574-fe8f-4846-844d-9182e3217ceb&response=json&sessionkey=J5RF%2BKuEucFSQi3XIQmJhxorqOM%3D&_=1368052554491 2013-05-08 15:35:18,211 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-88:job-66) Executing org.apache.cloudstack.api.command.admin.vpc.DeletePrivateGatewayCmd for job-66 2013-05-08 15:35:18,217 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-88:job-66) Sync job-66 execution on object vpc.3 2013-05-08 15:35:18,222 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-88:job-66) job org.apache.cloudstack.api.command.admin.vpc.DeletePrivateGatewayCmd for job-66 was queued, processing the queue. 2013-05-08 15:35:18,228 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-88:job-66) Executing sync queue item: SyncQueueItemVO {id:34, queueId: 32, contentType: AsyncJob, contentId: 66, lastProcessMsid: 7508777239729, lastprocessNumber: 3, lastProcessTime: Wed May 08 15:35:18 PDT 2013, created: Wed May 08 15:35:18 PDT 2013} 2013-05-08 15:35:18,229 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-88:job-66) Schedule queued job-66 2013-05-08 15:35:18,233 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-88:job-66) There is a pending process in sync queue(id: 32) 2013-05-08 15:35:18,234 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-89:job-66) Executing org.apache.cloudstack.api.command.admin.vpc.DeletePrivateGatewayCmd for job-66 2013-05-08 15:35:18,241 DEBUG [network.vpc.VpcManagerImpl] (Job-Executor-89:job-66) Marked gateway VpcGateway[14|10.223.60.34|3] with state Deleting 2013-05-08 15:35:18,251 DEBUG [network.router.VpcVirtualNetworkApplianceManagerImpl] (Job-Executor-89:job-66) Releasing private ip for gateway com.cloud.network.vpc.PrivateGatewayProfile@11cb0906 from VM[DomainRouter|r-7-NATONPRVGW] 2013-05-08 15:35:18,259 DEBUG [cloud.network.NetworkModelImpl] (Job-Executor-89:job-66) Service SecurityGroup is not supported in the network id=209 2013-05-08 15:35:18,270 DEBUG [agent.transport.Request] (Job-Executor-89:job-66) Seq 1-275056149: Sending { Cmd , MgmtId: 7508777239729, via: 1, Ver: v1, Flags: 100011, [{"routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":1,"publicIp":"10.223.60.35","sourceNat":true,"add":false,"oneToOneNat":false,"firstIP":false,"vlanId":"600","vlanGateway":"10.223.60.1","vlanNetmask":"255.255.255.192","vifMacAddress":"06:fd:40:00:00:2f","trafficType":"Guest"}],"accessDetails":{"router.guest.ip":"10.223.60.35","zone.network.type":"Advanced","router.ip":"169.254.0.221","router.name":"r-7-NATONPRVGW"},"wait":0}}] } 2013-05-08 15:35:18,641 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 1-275056149: Processing: { Ans: , MgmtId: 7508777239729, via: 1, Ver: v1, Flags: 10, [{"routing.IpAssocAnswer":{"results":["10.223.60.35 - success"],"result":true,"wait":0}}] } 2013-05-08 15:35:18,642 DEBUG [agent.transport.Request] (Job-Executor-89:job-66) Seq 1-275056149: Received: { Ans: , MgmtId: 7508777239729, via: 1, Ver: v1, Flags: 10, { IpAssocAnswer } } 2013-05-08 15:35:18,642 DEBUG [network.router.VpcVirtualNetworkApplianceManagerImpl] (Job-Executor-89:job-66) Successfully applied ip association for ip com.cloud.network.vpc.PrivateIpAddress@6425a347 in vpc network Ntwk[209|Guest|5] 2013-05-08 15:35:18,642 DEBUG [network.router.VpcVirtualNetworkApplianceManagerImpl] (Job-Executor-89:job-66) Removing router VM[DomainRouter|r-7-NATONPRVGW] from private network Ntwk[209|Guest|5] as a part of delete private gateway 2013-05-08 15:35:18,656 DEBUG [cloud.network.NetworkModelImpl] (Job-Executor-89:job-66) Service SecurityGroup is not supported in the network id=209 2013-05-08 15:35:18,658 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-89:job-66) Un-plugging nic for vm VM[DomainRouter|r-7-NATONPRVGW] from network Ntwk[209|Guest|5] 2013-05-08 15:35:18,660 DEBUG [agent.transport.Request] (Job-Executor-89:job-66) Seq 1-275056150: Sending { Cmd , MgmtId: 7508777239729, via: 1, Ver: v1, Flags: 100111, [{"UnPlugNicCommand":{"nic":{"deviceId":3,"networkRateMbps":200,"defaultNic":false,"uuid":"28516266-5d1f-449a-866a-bf42b110738b","ip":"10.223.60.35","netmask":"255.255.255.192","gateway":"10.223.60.1","mac":"06:fd:40:00:00:2f","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://600","isolationUri":"vlan://600","isSecurityGroupEnabled":false},"instanceName":"r-7-NATONPRVGW","wait":0}}] } 2013-05-08 15:35:18,906 DEBUG [agent.transport.Request] (AgentManager-Handler-4:null) Seq 1-275056150: Processing: { Ans: , MgmtId: 7508777239729, via: 1, Ver: v1, Flags: 110, [{"UnPlugNicAnswer":{"result":true,"details":"success","wait":0}}] } 2013-05-08 15:35:18,906 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-4:null) Seq 1-275056150: No more commands found 2013-05-08 15:35:18,906 DEBUG [agent.transport.Request] (Job-Executor-89:job-66) Seq 1-275056150: Received: { Ans: , MgmtId: 7508777239729, via: 1, Ver: v1, Flags: 110, { UnPlugNicAnswer } } mysql> select * from vpc_gateways where uuid="b6fe4574-fe8f-4846-844d-9182e3217ceb"; +----+--------------------------------------+--------------+-----------------+-------------+----------+---------+------------+--------+---------+---------------------+------------+-----------+----------+---------------------+------------+ | id | uuid | ip4_address | netmask | gateway | vlan_tag | type | network_id | vpc_id | zone_id | created | account_id | domain_id | state | removed | source_nat | +----+--------------------------------------+--------------+-----------------+-------------+----------+---------+------------+--------+---------+---------------------+------------+-----------+----------+---------------------+------------+ | 14 | b6fe4574-fe8f-4846-844d-9182e3217ceb | 10.223.60.34 | 255.255.255.192 | 10.223.60.1 | 600 | Private | 209 | 3 | 1 | 2013-05-08 22:27:58 | 2 | 1 | Deleting | 2013-05-08 22:35:18 | 1 | +----+--------------------------------------+--------------+-----------------+-------------+----------+---------+------------+--------+---------+---------------------+------------+-----------+----------+---------------------+------------+ 1 row in set (0.00 sec) mysql> select * from vpc_gateways where ip4_address like "10.223.60.35"; +----+--------------------------------------+--------------+-----------------+-------------+----------+---------+------------+--------+---------+---------------------+------------+-----------+-------+---------+------------+ | id | uuid | ip4_address | netmask | gateway | vlan_tag | type | network_id | vpc_id | zone_id | created | account_id | domain_id | state | removed | source_nat | +----+--------------------------------------+--------------+-----------------+-------------+----------+---------+------------+--------+---------+---------------------+------------+-----------+-------+---------+------------+ | 15 | 8e38b663-c43f-41fd-95cc-46571c3bffc8 | 10.223.60.35 | 255.255.255.192 | 10.223.60.1 | 600 | Private | 209 | 3 | 1 | 2013-05-08 22:28:13 | 2 | 1 | Ready | NULL | 1 | +----+--------------------------------------+--------------+-----------------+-------------+----------+---------+------------+--------+---------+---------------------+------------+-----------+-------+---------+------------+ 1 row in set (0.00 sec) root@r-7-NATONPRVGW:~# ifconfig eth0 Link encap:Ethernet HWaddr 0e:00:a9:fe:00:dd inet addr:169.254.0.221 Bcast:169.254.255.255 Mask:255.255.0.0 inet6 addr: fe80::c00:a9ff:fefe:dd/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:598 errors:0 dropped:0 overruns:0 frame:0 TX packets:553 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:72290 (70.5 KiB) TX bytes:91270 (89.1 KiB) eth1 Link encap:Ethernet HWaddr 06:fb:c0:00:00:0d inet addr:10.223.136.132 Bcast:10.223.136.191 Mask:255.255.255.192 inet6 addr: fe80::4fb:c0ff:fe00:d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:29 errors:0 dropped:0 overruns:0 frame:0 TX packets:39 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2602 (2.5 KiB) TX bytes:3150 (3.0 KiB) eth2 Link encap:Ethernet HWaddr 06:db:b0:00:00:2e inet addr:10.223.60.34 Bcast:10.223.60.63 Mask:255.255.255.192 inet6 addr: fe80::4db:b0ff:fe00:2e/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:23 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1386 (1.3 KiB) TX bytes:444 (444.0 B) eth3 Link encap:Ethernet HWaddr 06:87:2c:00:00:30 inet addr:10.223.60.36 Bcast:10.223.60.63 Mask:255.255.255.192 inet6 addr: fe80::487:2cff:fe00:30/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:714 (714.0 B) TX bytes:444 (444.0 B) eth4 Link encap:Ethernet HWaddr 06:65:44:00:00:31 inet addr:10.223.60.37 Bcast:10.223.60.63 Mask:255.255.255.192 inet6 addr: fe80::465:44ff:fe00:31/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:7 errors:0 dropped:0 overruns:0 frame:0 TX packets:7 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:510 (510.0 B) TX bytes:514 (514.0 B) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:8 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1104 (1.0 KiB) TX bytes:1104 (1.0 KiB) root@r-7-NATONPRVGW:~# -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira