Re: VPC ACLs and Loadbalancer

[Andrija Panic]

Well :) that is a good question - desired by who :)

ACLs are applied on routined traffic (i.e. traffic between networks), so
here its simply not aplicable - you connect to LOCAL port/service on VR
(imagine port 22 as in mine example, but otherwise default rules are all
DENY, so you can't access haproxy if you just i.e. manually install and
configure haproxy listener on port 22 - but when you do it via GUI,
additional rule is added to iptables, to allow actually this connect (to
port 22 = -A INPUT -d 185.39.xxx.yyy/32 -p tcp -m tcp --dport 22 -m
state --state NEW -j ACCEPT )

So if you enabled LB serices on specific port, ACS will make sure it's
allowed inside iptables...
For inter-traffic network, you you classic ACL.

Hope that help

Andrija

On 14 February 2018 at 11:34, S. Reddit  wrote:

> Hi Andrija
>
> Wow - thanks for in-depth analysis! I already suspected HAProxy services
> not hitting iptables chain.
>
> Thanks for clarification, I see that the behaviour is EXPECTED, is it also
> DESIRED?
>
> Regards,
> Samuel
>



-- 

Andrija Panić

Re: VPC ACLs and Loadbalancer

[S. Reddit]

Hi Andrija

Wow - thanks for in-depth analysis! I already suspected HAProxy services
not hitting iptables chain.

Thanks for clarification, I see that the behaviour is EXPECTED, is it also
DESIRED?

Regards,
Samuel

Re: VPC ACLs and Loadbalancer

[Andrija Panic]

Hi S,

so I have reproduced same behavior on ACS 4.8.x and from what I can see
this is EXPECTED for following reason:

root@r-4997-VM:~# iptables-save | grep  "\-j ACL"
 -A PREROUTING -s 10.10.10.0/24 ! -d 10.10.10.1/32 -i eth2 -m state
--state NEW -j ACL_OUTBOUND_eth2
 -A FORWARD -d 10.10.10.0/24 -o eth2 -j ACL_INBOUND_eth2

Here above there is single network inside VPC, 10.10.10.0/24 with
10.10.10.1 IP being the IP inside VR (gtw for VMs in this networkm eth2
inside VR)

So above you see that only traffic that is in PREROUTING/FORWARD chain will
end up being forwarded to ACL chains (evaluated by ACL rules)  (not even
all prerouting/forward traffic, but just the with specific source IP, as
you defined in your ACL)
...

But for Loadbalancing via Haproxy (I did LB of port 22, unconventional, but
for test), here is what is inside iptables:

   -A INPUT -d 185.39.xxx.yyy/32 -p tcp -m tcp --dport 22 -m state
--state NEW -j ACCEPT

THis is the iptables rule, so connections to this port are on INPUT chain,
not inside PREROUTING or FORWARD... so the traffic is NOT inspected at all
by ACL rules in chain  ACL_OUTBOUND_eth2 or  ACL_INBOUND_eth2

So I hope that is clear.

Furhter more, when you connect to haproxy public IP (the rule which accept
traffic, as mentioned above is  -A INPUT -d 185.39.xxx.yyy/32 -p tcp -m tcp
--dport 22 -m state --state NEW -j ACCEPT) -  so the connection is accepted
to the public IP (listener port 22 on local VR)

Next, the haproxy opens the connection on it's own, from it's private ip
address 10.10.10.1 to the internal VM 10.10.10.10, as shown below

a) relevant haproxy config

  listen 185_39_229_176-22 185.39.229.176:22
 server 185_39_229_176-22_0 10.10.10.10:22 check

b) netstat -antup | grep 10.10.10.10
  tcp0  0 10.10.10.1:5613210.10.10.10:22
ESTABLISHED 3777/haproxy

Soagain,  the way Haprox works is NOT routing of any sense, and thus ACLs
are not applied - here VR will accept traffic via INPUT on its public IP -
PERIOD, no more packet flow this packet, it stops here. Next step, the
haproxy will open NEW TCP CONNECTION to internal VM, via OUTPUT chain which
is set to ALLOW by default (iptables-save | grep OUTPUT).

Irelevant to your problem:

So you see that for PREROUTING and FORWARD chains, the traffic is
redirected to the ACL (PREROUTNG go to OUTBOUND part of ACL, and FORWARD go
to the INBOUND part of ACL - inside VR these are 2 chains, one for outgoing
traffic and the other one for incoming traffic to the network - in other
words all rules inside same/single/one ACL in ACS, will get split into this
2 chains, - IRELEVANT HERE FOR THE PROBLEM you have, just expaininig...:)

Cheers
Andrija


On 12 February 2018 at 13:28, S. Reddit  wrote:

> Hi List
>
> We face an issue with VPC and ACLs together with Loadbalancing (on
> vRouter). The ACL rules do not seem to work at all. Steps to reproduce:
> - Create a VPC
> - Create Tier with Public LB Services on vRouter
> - Apply default_deny ACL
> - Create Instance
> - Create Public LB-Rule on Public IP and point to instance
>
> => VM is accessable via LB-IP, although ACL is set to default_deny.
>
> CloudStack Version is 4.9.2 Anyone seen this as well? The behaviour with
> Static-Nat and Port-Forward is as expected.
>
> Regards,
> S
>



-- 

Andrija Panić

VPC ACLs and Loadbalancer

[S. Reddit]

Hi List

We face an issue with VPC and ACLs together with Loadbalancing (on
vRouter). The ACL rules do not seem to work at all. Steps to reproduce:
- Create a VPC
- Create Tier with Public LB Services on vRouter
- Apply default_deny ACL
- Create Instance
- Create Public LB-Rule on Public IP and point to instance

=> VM is accessable via LB-IP, although ACL is set to default_deny.

CloudStack Version is 4.9.2 Anyone seen this as well? The behaviour with
Static-Nat and Port-Forward is as expected.

Regards,
S