venkata swamybabu budumuru created CLOUDSTACK-4084:
------------------------------------------------------
Summary: [MultipleIPsPerNic] [ExternalDevices] After Network GC,
staticNat is not working If the staticNat is on primary guest IP
Key: CLOUDSTACK-4084
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4084
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Network Controller
Affects Versions: 4.2.0
Environment: commit #
Reporter: venkata swamybabu budumuru
Assignee: Jayapal Reddy
Priority: Critical
Fix For: 4.2.0
Attachments: logs.tgz
Steps to reproduce:
1. Have latest CloudStack setup with at least 1 adv zone using KVM hosts
2. Make sure network.gc.interval and wait are set to "100" seconds
3. Have at least one network offering of type "isolated" and with all services
enabled where LB is provided by NS and other services are provided by VR.
mysql> select * from network_offerings where id=15\G
*************************** 1. row ***************************
id: 15
name: NetworkOffering with NS
uuid: 4aaf5c58-6d45-4213-8c26-0b2b6f6792c5
unique_name: NetworkOffering with NS
display_text: NetworkOffering with NS
nw_rate: NULL
mc_rate: 10
traffic_type: Guest
tags: NULL
system_only: 0
specify_vlan: 0
service_offering_id: NULL
conserve_mode: 0
created: 2013-08-05 07:30:38
removed: NULL
default: 0
availability: Optional
dedicated_lb_service: 0
shared_source_nat_service: 0
sort_key: 0
redundant_router_service: 0
state: Enabled
guest_type: Isolated
elastic_ip_service: 0
eip_associate_public_ip: 0
elastic_lb_service: 0
specify_ip_ranges: 0
inline: 0
is_persistent: 0
internal_lb: 0
public_lb: 1
egress_default_policy: 1
concurrent_connections: NULL
mysql> select * from ntwk_offering_service_map where network_offering_id=15;
+----+---------------------+----------------+---------------+---------------------+
| id | network_offering_id | service | provider | created |
+----+---------------------+----------------+---------------+---------------------+
| 58 | 15 | Dhcp | VirtualRouter | 2013-08-05 07:30:38 |
| 55 | 15 | Dns | VirtualRouter | 2013-08-05 07:30:38 |
| 60 | 15 | Firewall | VirtualRouter | 2013-08-05 07:30:38 |
| 59 | 15 | Lb | Netscaler | 2013-08-05 07:30:38 |
| 54 | 15 | PortForwarding | VirtualRouter | 2013-08-05 07:30:38 |
| 56 | 15 | SourceNat | VirtualRouter | 2013-08-05 07:30:38 |
| 53 | 15 | StaticNat | VirtualRouter | 2013-08-05 07:30:38 |
| 57 | 15 | UserData | VirtualRouter | 2013-08-05 07:30:38 |
| 61 | 15 | Vpn | VirtualRouter | 2013-08-05 07:30:38 |
+----+---------------------+----------------+---------------+---------------------+
4. deploy a VM using the network that is created using above offering and then
create the following rules
NOTE : Guest got a primary address as 10.0.0.62/20
(i) Create PF rule that maps to the primary guest ip
mysql> select * from user_ip_address where public_ip_address like
'%10.147.44.65%'\G
*************************** 1. row ***************************
id: 6
uuid: f6a454d8-ab7f-4b3b-a763-f6cf1a14b219
account_id: 3
domain_id: 2
public_ip_address: 10.147.44.65
data_center_id: 1
source_nat: 0
allocated: 2013-08-05 09:51:26
vlan_db_id: 1
one_to_one_nat: 0
vm_id: NULL
state: Allocated
mac_address: 16
source_network_id: 200
network_id: 210
physical_network_id: 200
is_system: 0
vpc_id: NULL
dnat_vmip: NULL
is_portable: 0
(ii) Create a staticNat rule that maps to the primary guest ip
mysql> select * from user_ip_address where public_ip_address like
'%10.147.44.64%'\G
*************************** 1. row ***************************
id: 5
uuid: 16ed5cfa-795d-48a1-b11d-7c6fe3f2bbd7
account_id: 3
domain_id: 2
public_ip_address: 10.147.44.64
data_center_id: 1
source_nat: 0
allocated: 2013-08-05 09:48:05
vlan_db_id: 1
one_to_one_nat: 1
vm_id: 10
state: Allocated
mac_address: 15
source_network_id: 200
network_id: 210
physical_network_id: 200
is_system: 0
vpc_id: NULL
dnat_vmip: 10.0.0.62
is_portable: 0
1 row in set (0.00 sec)
(iii) Initially for the above network the CIDR given to it is : 10.0.0.0/20
5. Make sure all the above rules are working. Now, power off the userVM and
wait till network GC
Observations:
(i) After GC, the above network got 10.0.80.0/20 as CIDR and everything worked
fine without issues except StatiNat rule.
(ii) When I tried to ssh to 10.147.44.64 then it failed. check the VR for the
rule and found that it is still programming staticNat with old guest ip i.e.
10.0.0.62 instead of new guest ip i.e. 10.0.80.62/20.
Note : PF went fine without any issues.
here is the snippet from VR.
root@r-15-VM:~# iptables -L -nv -t nat
Chain PREROUTING (policy ACCEPT 40 packets, 4127 bytes)
pkts bytes target prot opt in out source destination
0 0 DNAT all -- eth2 * 0.0.0.0/0
10.147.44.64 to:10.0.0.62
0 0 DNAT all -- eth0 * 0.0.0.0/0
10.147.44.64 to:10.0.0.62
0 0 DNAT tcp -- eth2 * 0.0.0.0/0
10.147.44.65 tcp dpt:22 to:10.0.80.62:22
0 0 DNAT tcp -- eth0 * 0.0.0.0/0
10.147.44.65 tcp dpt:22 to:10.0.80.62:22
Chain INPUT (policy ACCEPT 23 packets, 1882 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 1 packets, 341 bytes)
pkts bytes target prot opt in out source destination
0 0 DNAT tcp -- * * 0.0.0.0/0
10.147.44.65 tcp dpt:22 to:10.0.80.62:22
Chain POSTROUTING (policy ACCEPT 1 packets, 341 bytes)
pkts bytes target prot opt in out source destination
0 0 SNAT all -- * eth2 10.0.0.62 0.0.0.0/0
to:10.147.44.64
0 0 SNAT all -- * eth2 0.0.0.0/0 0.0.0.0/0
to:10.147.44.63
0 0 SNAT all -- * eth0 10.0.80.0/20 10.0.0.62
to:10.0.80.1
0 0 SNAT tcp -- * eth0 10.0.80.0/20 10.0.80.62
tcp dpt:22 to:10.0.80.1
(iii) After GC, when I checked the user_ip_address table then I found that it
still has dnat_vmip still set to "10.0.0.62" and I am doubting that this is
causing the issue. Ideally we should have cleaned this during gC and should be
setting this during rule reprogramming.
Attaching all the required logs along with db dump to the bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
