[
https://issues.apache.org/jira/browse/CLOUDSTACK-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16125343#comment-16125343
]
Francois Scheurer commented on CLOUDSTACK-10043:
------------------------------------------------
Hello Rafael
Yes all VR were recreated.
We are using the 4.6.0 Version: root@r-1950-VM:~# cat /etc/cloudstack-release
Cloudstack Release 4.6.0 Mon Dec 19 09:59:35 UTC 2016
We used the link
http://packages.shapeblue.com/systemvmtemplate/4.6/new/systemvm64template-4.6-xen.vhd.bz2
from the docu here
http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.9.2.0/upgrade/upgrade-4.5.html
.
Do you think we should use another image?
We created the Rules via the ACS API.
Some of them were then present in the iptables filter table and some in mangle.
AFAIK all should be in filter and none in mangle.
And most importantly, the precedence numbers (in DB:
network_acl_item_item.number) are not respected in the VR.
Below are the rules from DB and from the Iptables to let you compare.
I will create a new VPC Offering and Network Offering from scratch and then
create a new VPC with some egress rules and post the results.
Best Regards
Francois Scheurer
ACL:
mysql> select * from network_acl_item left join network_acl_item_cidrs on
network_acl_item.id=network_acl_item_cidrs.network_acl_item_id where
network_acl_item.state="Active" and acl_id=(select network_acl_id from networks
where id=388);
+------+--------------------------------------+--------+------------+----------+--------+----------+---------------------+-----------+-----------+--------------+--------+--------+---------+------+---------------------+-------------------+
| id | uuid | acl_id | start_port |
end_port | state | protocol | created | icmp_code | icmp_type |
traffic_type | number | action | display | id | network_acl_item_id | cidr
|
+------+--------------------------------------+--------+------------+----------+--------+----------+---------------------+-----------+-----------+--------------+--------+--------+---------+------+---------------------+-------------------+
| 2620 | 99cf33db-64fc-4708-be2b-fdc150300645 | 140 | 123 |
123 | Active | udp | 2016-04-06 19:29:27 | NULL | NULL | Egress
| 40 | Allow | 1 | 2988 | 2620 |
AAA.BBB.111.18/32 |
| 2596 | f06f7fdb-e68f-48c3-bbfa-1223c88e2f7b | 140 | 80 |
80 | Active | tcp | 2016-04-06 07:43:49 | NULL | NULL | Egress
| 50 | Allow | 1 | 2980 | 2596 | AAA.BBB.55.80/32
|
| 2600 | afefcab3-89db-41d4-9ebe-89680f70416b | 140 | 80 |
80 | Active | tcp | 2016-04-06 07:44:42 | NULL | NULL | Egress
| 51 | Allow | 1 | 2956 | 2600 | AAA.BBB.9.40/32
|
| 2604 | e65628b6-f050-4e17-b1f9-dbaba8a29e7c | 140 | 80 |
80 | Active | tcp | 2016-04-06 07:45:13 | NULL | NULL | Egress
| 52 | Allow | 1 | 2984 | 2604 |
AAA.BBB.118.125/32 |
| 2608 | 4b900d29-9eb7-4c7f-a2ff-15a94d35d2b3 | 140 | 80 |
80 | Active | tcp | 2016-04-06 07:45:51 | NULL | NULL | Egress
| 53 | Allow | 1 | 2968 | 2608 | AAA.BBB.55.80/32
|
| 2612 | b38940e4-96b7-4f4b-b581-f0a8d5651e7a | 140 | 80 |
80 | Active | tcp | 2016-04-06 07:46:46 | NULL | NULL | Egress
| 54 | Allow | 1 | 2972 | 2612 | AAA.BBB.79.66/32
|
| 2616 | 9f87246c-097c-414a-8320-e68905db6b95 | 140 | 80 |
80 | Active | tcp | 2016-04-06 07:47:51 | NULL | NULL | Egress
| 55 | Allow | 1 | 2976 | 2616 |
AAA.BBB.116.80/32 |
| 1044 | 09e83ceb-b2a8-4b1e-99e9-233b8449ac30 | 140 | 443 |
443 | Active | tcp | 2015-11-09 14:23:49 | NULL | NULL | Egress
| 100 | Allow | 1 | 1072 | 1044 |
AAA.BBB.134.1/32 |
| 1064 | 2e3c12b1-3222-4b27-956e-0811a157256b | 140 | 443 |
443 | Active | tcp | 2015-11-09 14:42:44 | NULL | NULL | Egress
| 110 | Allow | 1 | 1092 | 1064 |
AAA.BBB.134.2/32 |
| 1068 | 67084ffe-f419-4de8-b157-c92d4a2f8c6f | 140 | 3128 |
3128 | Active | tcp | 2015-11-09 14:47:20 | NULL | NULL |
Ingress | 150 | Allow | 1 | 1096 | 1068 |
0.0.0.0/0 |
| 1048 | 7e44dc24-fba0-4e8e-9cd9-3aa26237e51f | 140 | 22 |
22 | Active | tcp | 2015-11-09 14:24:11 | NULL | NULL | Ingress
| 200 | Allow | 1 | 1076 | 1048 | AAA.BBB.117.0/25
|
| 1052 | c866a5c8-b5ff-4c57-9644-dda5c703ca5f | 140 | 25 |
25 | Active | tcp | 2015-11-09 14:25:04 | NULL | NULL | Egress
| 300 | Allow | 1 | 1080 | 1052 |
AAA.BBB.111.45/32 |
| 1056 | 979114be-0b6f-44fc-9dd8-29ef93a77595 | 140 | NULL |
NULL | Active | icmp | 2015-11-09 14:33:33 | 0 | 8 |
Ingress | 400 | Allow | 1 | 1084 | 1056 |
0.0.0.0/0 |
| 1060 | d43a4d90-da14-476f-8a03-10c3d28817d2 | 140 | NULL |
NULL | Active | icmp | 2015-11-09 14:34:01 | 4 | 3 |
Ingress | 500 | Allow | 1 | 1088 | 1060 |
0.0.0.0/0 |
| 1500 | b05209b0-2dae-4ee7-8e37-632f7d843c03 | 140 | 5666 |
5666 | Active | tcp | 2015-12-10 17:11:43 | NULL | NULL |
Ingress | 600 | Allow | 1 | 1564 | 1500 |
AAA.BBB.125.220/32 |
| 1504 | 208c26ca-9563-4742-abae-6ed79b642177 | 140 | 22 |
22 | Active | tcp | 2015-12-10 17:12:03 | NULL | NULL | Ingress
| 700 | Allow | 1 | 1568 | 1504 |
AAA.BBB.125.220/32 |
| 1508 | 7b7f716f-86ba-4aae-a87c-06ee68c07a20 | 140 | 3128 |
3128 | Active | tcp | 2015-12-10 17:14:35 | NULL | NULL |
Ingress | 800 | Allow | 1 | 1572 | 1508 |
AAA.BBB.125.220/32 |
| 1188 | c4e6f89b-4b83-45aa-8cc3-1def7ad8c88d | 140 | 443 |
443 | Active | tcp | 2015-11-27 09:37:41 | NULL | NULL | Egress
| 1000 | Allow | 1 | 1252 | 1188 |
AAA.BBB.71.201/32 |
| 1637 | 7d6b7dd1-67da-4d7a-92b8-9b6f52757492 | 140 | 80 |
80 | Active | tcp | 2016-01-12 10:28:34 | NULL | NULL | Egress
| 1100 | Allow | 1 | 1713 | 1637 |
AAA.BBB.111.13/32 |
| 1876 | 6165879a-48af-48fa-9fee-85e0f6a670c1 | 140 | 80 |
80 | Active | tcp | 2016-02-25 09:02:41 | NULL | NULL | Egress
| 1200 | Allow | 1 | 2112 | 1876 |
AAA.BBB.112.123/32 |
| 1880 | 6a6571b7-ed32-432a-8cd5-07319ca53d60 | 140 | 290 |
291 | Active | tcp | 2016-02-25 09:03:16 | NULL | NULL | Ingress
| 1300 | Allow | 1 | 2116 | 1880 |
AAA.BBB.112.123/32 |
| 4155 | 13bfa058-2531-45af-9198-dbd86404ad07 | 140 | NULL |
NULL | Active | all | 2017-08-08 08:05:19 | NULL | NULL | Egress
| 1400 | Allow | 1 | 4738 | 4155 | 0.0.0.0/0
|
+------+--------------------------------------+--------+------------+----------+--------+----------+---------------------+-----------+-----------+--------------+--------+--------+---------+------+---------------------+-------------------+
Iptables with corresponding precedence number on the left (it is not an
increasing serie as it should be):
# Generated by iptables-save v1.4.14 on Mon Aug 7 16:08:16 2017
*mangle
:PREROUTING ACCEPT [4:1184]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [4:1184]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [4:1184]
:ACL_OUTBOUND_eth2 - [0:0]
:VPN_STATS_eth1 - [0:0]
-A PREROUTING -i eth2 -m state --state RELATED,ESTABLISHED -j CONNMARK
--restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A PREROUTING -s 10.4.96.0/27 ! -d 10.4.96.1/32 -i eth2 -m state
--state NEW -j ACL_OUTBOUND_eth2
-A PREROUTING -i eth1 -m state --state NEW -j CONNMARK --set-xmark
0x1/0xffffffff
-A PREROUTING -s 10.4.96.9/32 -m state --state NEW -j MARK --set-xmark
0x1/0xffffffff
-A PREROUTING -s 10.4.96.9/32 -m state --state NEW -j CONNMARK
--save-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A FORWARD -j VPN_STATS_eth1
-A OUTPUT -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
-A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
-A ACL_OUTBOUND_eth2 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth2 -d 225.0.0.50/32 -j ACCEPT
1400 -A ACL_OUTBOUND_eth2 -j DROP
40 -A ACL_OUTBOUND_eth2 -d AAA.BBB.111.18/32 -p udp -m udp --dport 123 -j
ACCEPT
50 -A ACL_OUTBOUND_eth2 -d AAA.BBB.55.80/32 -p tcp -m tcp --dport 80 -j
ACCEPT
51 -A ACL_OUTBOUND_eth2 -d AAA.BBB.9.40/32 -p tcp -m tcp --dport 80 -j
ACCEPT
52 -A ACL_OUTBOUND_eth2 -d AAA.BBB.118.125/32 -p tcp -m tcp --dport 80 -j
ACCEPT
53 -A ACL_OUTBOUND_eth2 -d AAA.BBB.55.80/32 -p tcp -m tcp --dport 80 -j
ACCEPT
54 -A ACL_OUTBOUND_eth2 -d AAA.BBB.79.66/32 -p tcp -m tcp --dport 80 -j
ACCEPT
55 -A ACL_OUTBOUND_eth2 -d AAA.BBB.116.80/32 -p tcp -m tcp --dport 80 -j
ACCEPT
100 -A ACL_OUTBOUND_eth2 -d AAA.BBB.134.1/32 -p tcp -m tcp --dport 443 -j
ACCEPT
110 -A ACL_OUTBOUND_eth2 -d AAA.BBB.134.2/32 -p tcp -m tcp --dport 443 -j
ACCEPT
300 -A ACL_OUTBOUND_eth2 -d AAA.BBB.111.45/32 -p tcp -m tcp --dport 25 -j
ACCEPT
1000 -A ACL_OUTBOUND_eth2 -d AAA.BBB.71.201/32 -p tcp -m tcp --dport 443 -j
ACCEPT
1100 -A ACL_OUTBOUND_eth2 -d AAA.BBB.111.13/32 -p tcp -m tcp --dport 80 -j
ACCEPT
1200 -A ACL_OUTBOUND_eth2 -d AAA.BBB.112.123/32 -p tcp -m tcp --dport 80 -j
ACCEPT
-A VPN_STATS_eth1 -o eth1 -m mark --mark 0x525
-A VPN_STATS_eth1 -i eth1 -m mark --mark 0x524
COMMIT
# Completed on Mon Aug 7 16:08:16 2017
# Generated by iptables-save v1.4.14 on Mon Aug 7 16:08:16 2017
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -d 217.71.94.144/32 -i eth0 -j DNAT --to-destination
10.4.96.9
-A PREROUTING -d 217.71.94.144/32 -j DNAT --to-destination 10.4.96.9
-A OUTPUT -d 217.71.94.144/32 -j DNAT --to-destination 10.4.96.9
-A POSTROUTING -s 10.4.96.0/27 -d 10.4.96.9/32 -o eth0 -j SNAT
--to-source 10.4.96.1
-A POSTROUTING -s 10.4.96.9/32 -o eth1 -j SNAT --to-source 217.71.94.144
-A POSTROUTING -s 10.4.96.0/27 -o eth2 -j SNAT --to-source 10.4.96.1
-A POSTROUTING -o eth1 -j SNAT --to-source 217.71.94.151
COMMIT
# Completed on Mon Aug 7 16:08:16 2017
# Generated by iptables-save v1.4.14 on Mon Aug 7 16:08:16 2017
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:ACL_INBOUND_eth2 - [0:0]
:FW_EGRESS_RULES - [0:0]
:NETWORK_STATS - [0:0]
:NETWORK_STATS_eth1 - [0:0]
-A INPUT -d 10.4.96.1/32 -i eth2 -p tcp -m tcp --dport 443 -m state
--state NEW -j ACCEPT
-A INPUT -d 10.4.96.1/32 -i eth2 -p tcp -m tcp --dport 80 -m state
--state NEW -j ACCEPT
-A INPUT -d 10.4.96.1/32 -i eth2 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -d 10.4.96.1/32 -i eth2 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -j NETWORK_STATS
-A INPUT -i eth2 -p udp -m udp --dport 67 -j ACCEPT
-A INPUT -s 10.4.96.0/27 -i eth2 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -s 10.4.96.0/27 -i eth2 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i eth2 -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
-A INPUT -i eth2 -p tcp -m tcp --dport 8080 -m state --state NEW -j
ACCEPT
-A INPUT -d 224.0.0.18/32 -j ACCEPT
-A INPUT -d 225.0.0.50/32 -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -i eth0 -p tcp -m tcp --dport 3922 -m state --state
NEW,ESTABLISHED -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j NETWORK_STATS_eth1
-A FORWARD -j NETWORK_STATS
-A FORWARD -s 10.4.96.0/24 ! -d 10.4.96.0/24 -j ACCEPT
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -d 10.4.96.0/27 -o eth2 -j ACL_INBOUND_eth2
-A FORWARD -d 10.4.96.9/32 -i eth1 -o eth0 -m state --state NEW -j
ACCEPT
-A OUTPUT -j NETWORK_STATS
-A ACL_INBOUND_eth2 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth2 -d 224.0.0.18/32 -j ACCEPT
150 -A ACL_INBOUND_eth2 -p tcp -m tcp --dport 3128 -j ACCEPT
200 -A ACL_INBOUND_eth2 -s AAA.BBB.117.0/25 -p tcp -m tcp --dport 22 -j
ACCEPT
400 -A ACL_INBOUND_eth2 -p icmp -m icmp --icmp-type 8/0 -j ACCEPT
500 -A ACL_INBOUND_eth2 -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
600 -A ACL_INBOUND_eth2 -s AAA.BBB.125.220/32 -p tcp -m tcp --dport 5666 -j
ACCEPT
700 -A ACL_INBOUND_eth2 -s AAA.BBB.125.220/32 -p tcp -m tcp --dport 22 -j
ACCEPT
800 -A ACL_INBOUND_eth2 -s AAA.BBB.125.220/32 -p tcp -m tcp --dport 3128 -j
ACCEPT
1300 -A ACL_INBOUND_eth2 -s AAA.BBB.112.123/32 -p tcp -m tcp --dport
290:291 -j ACCEPT
-A ACL_INBOUND_eth2 -j DROP
-A NETWORK_STATS -i eth0 -o eth2 -p tcp
-A NETWORK_STATS -i eth2 -o eth0 -p tcp
-A NETWORK_STATS ! -i eth0 -o eth2 -p tcp
-A NETWORK_STATS -i eth2 ! -o eth0 -p tcp
-A NETWORK_STATS_eth1 -s 10.4.96.0/27 -o eth1
-A NETWORK_STATS_eth1 -d 10.4.96.0/27 -i eth1
COMMIT
# Completed on Mon Aug 7 16:08:16 2017
> Egress Rule in VPC ACL broken
> ------------------------------
>
> Key: CLOUDSTACK-10043
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10043
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Virtual Router, VPC
> Affects Versions: 4.9.2.0
> Environment: CS 4.9.2 with XenServer 6.5SP1
> Reporter: Francois Scheurer
> Priority: Blocker
>
> The Network Offering of the VPC Tier has a Default Egress Policy = Deny.
> Some Allow Rules exist in the ACL, but _ALL_ egress connections are possible.
> Creating a Deny All rule explicit at the end of the rules is actually
> blocking ALL traffic (should not, because of the Allow rules).
> The Iptables in the VR are wrong:
> 1)the allow rules are in wrong order.
> 2)some rules are in mangle table instead of filter
> Do you know how to fix this?
> Thank you for your help.
> Francois Scheurer
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
