In general, the new ruleset looks good. I believe it fixes the issue at hand. After staring at the rules for a bit, I do have a few questions:
Why can't this: -A felix-from-8b91b3f4-0a -j MARK --set-xmark 0x0/0xffffffff -A felix-from-8b91b3f4-0a -s 10.47.3.2/32 -m mac --mac-source FA:16:3E:3A:C4:68 -j felix-p-_b28e0174207abac-o -A felix-from-8b91b3f4-0a -m mark ! --mark 0x1/0x1 -m comment --comment "No mark means profile accepted packet" -j RETURN -A felix-from-8b91b3f4-0a -j MARK --set-xmark 0x0/0xffffffff -A felix-from-8b91b3f4-0a -s 10.47.3.2/32 -m mac --mac-source FA:16:3E:3A:C4:68 -j felix-p-_1a49d8473c85194-o -A felix-from-8b91b3f4-0a -m mark ! --mark 0x1/0x1 -m comment --comment "No mark means profile accepted packet" -j RETURN -A felix-from-8b91b3f4-0a -m comment --comment "Default DROP if no match (endpoint 8b91b3f4-0ac8-4a2d-b81f-b184400bda15):" -j DROP -A felix-p-_b28e0174207abac-o -m comment --comment "Mark as not matched" -j MARK --set-xmark 0x1/0xffffffff -A felix-p-_1a49d8473c85194-o -j RETURN -A felix-p-_1a49d8473c85194-o -m comment --comment "Mark as not matched" -j MARK --set-xmark 0x1/0xffffffff -A felix-to-8b91b3f4-0a -j MARK --set-xmark 0x0/0xffffffff -A felix-to-8b91b3f4-0a -j felix-p-_b28e0174207abac-i -A felix-to-8b91b3f4-0a -m mark ! --mark 0x1/0x1 -m comment --comment "No mark means profile accepted packet" -j RETURN -A felix-to-8b91b3f4-0a -j MARK --set-xmark 0x0/0xffffffff -A felix-to-8b91b3f4-0a -j felix-p-_1a49d8473c85194-i -A felix-to-8b91b3f4-0a -m mark ! --mark 0x1/0x1 -m comment --comment "No mark means profile accepted packet" -j RETURN -A felix-to-8b91b3f4-0a -m comment --comment "Endpoint 8b91b3f4-0ac8-4a2d-b81f-b184400bda15:" -j DROP -A felix-p-_1a49d8473c85194-i -p tcp -m multiport --dports 22 -j RETURN -A felix-p-_1a49d8473c85194-i -p tcp -m multiport --dports 80 -j RETURN -A felix-p-_1a49d8473c85194-i -p icmp -j RETURN -A felix-p-_1a49d8473c85194-i -m set --match-set felix-v4-_1a49d8473c85194 src -j RETURN -A felix-p-_1a49d8473c85194-i -m comment --comment "Mark as not matched" -j MARK --set-xmark 0x1/0xffffffff -A felix-p-_b28e0174207abac-i -p tcp -m multiport --dports 9999 -j RETURN -A felix-p-_b28e0174207abac-i -m comment --comment "Mark as not matched" -j MARK --set-xmark 0x1/0xffffffff Be simplified to this? -A felix-from-8b91b3f4-0a -s 10.47.3.2/32 -m mac --mac-source FA:16:3E:3A:C4:68 -g felix-p-_b28e0174207abac-o -A felix-from-8b91b3f4-0a -s 10.47.3.2/32 -m mac --mac-source FA:16:3E:3A:C4:68 -g felix-p-_1a49d8473c85194-o -A felix-from-8b91b3f4-0a -m comment --comment "Default DROP if no match (endpoint 8b91b3f4-0ac8-4a2d-b81f-b184400bda15):" -j DROP -A felix-p-_1a49d8473c85194-o -j RETURN -A felix-to-8b91b3f4-0a -g felix-p-_b28e0174207abac-i -A felix-to-8b91b3f4-0a -g felix-p-_1a49d8473c85194-i -A felix-to-8b91b3f4-0a -m comment --comment "Endpoint 8b91b3f4-0ac8-4a2d-b81f-b184400bda15:" -j DROP -A felix-p-_1a49d8473c85194-i -p tcp -m multiport --dports 22 -j RETURN -A felix-p-_1a49d8473c85194-i -p tcp -m multiport --dports 80 -j RETURN -A felix-p-_1a49d8473c85194-i -p icmp -j RETURN -A felix-p-_1a49d8473c85194-i -m set --match-set felix-v4-_1a49d8473c85194 src -j RETURN -A felix-p-_b28e0174207abac-i -p tcp -m multiport --dports 9999 -j RETURN On the felix-INPUT chain, connection state rules are setup for all interfaces (as apposed to only hitting tap+ like all the others). I'm not sure if that's a good thing or not. Generally a working system will need something like that, but it would be nice if felix scoped all it's iptables rules to just the things it needs (e.g. tap+). Also, a lot of those connection tracking rules seem redundant. Instead of all the current instances of "--ctstate INVALID -j DROP" and "-m conntrack --ctstate RELATED,ESTABLISHED -j RETURN", can't we just do this at the top level? -A felix-FORWARD -i tap+ -m conntrack --ctstate INVALID -j DROP -A felix-FORWARD -i tap+ -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A felix-FORWARD -i tap+ -j felix-FROM-ENDPOINT -A felix-FORWARD -o tap+ -m conntrack --ctstate INVALID -j DROP -A felix-FORWARD -o tap+ -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A felix-FORWARD -o tap+ -j felix-TO-ENDPOINT -A felix-INPUT -i tap+ -m conntrack --ctstate INVALID -j DROP -A felix-INPUT -i tap+ -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT This unfortunately doesn't handle the case where a connection is in progress and an updated security policy is applied which would no longer allow the connection, as that connection would continue to operate. However the current ruleset does not appear to handle that case either. The entry: -A felix-INPUT -i tap+ -j RETURN Is redundant, as anything that falls off the end will return. Although I'm not sure it's worth special casing in the code. On Thu, Jun 18, 2015 at 2:01 PM, Nick Bartos <n...@pistoncloud.com> wrote: > We have routable floating IPs for the various services that get assigned > dynamically as aliases on a (non-loopback) device. No NAT is being done > for these currently. > > On Thu, Jun 18, 2015 at 1:57 PM, Shaun Crampton < > shaun.cramp...@metaswitch.com> wrote: > >> Sorry Nick, I think we missed that use-case when tightening up those >> rules but I think it’s something that we want to support. >> >> I’ve filed a Github issue to track this: >> https://github.com/Metaswitch/calico/issues/659. >> >> How are you exposing the services, are you running them on 127.0.0.1 >> (or 0.0.0.0) and then NATing or is the VM just talking to the host on its >> external IP address? >> >> Thanks, >> >> -Shaun >> >> From: Nick Bartos <n...@pistoncloud.com> >> Date: Thursday, 18 June 2015 13:30 >> To: "calico-tech@lists.projectcalico.org" < >> calico-tech@lists.projectcalico.org> >> Subject: Re: [calico-tech] Problem with new iptables rules in >> calico/master >> >> Actually now that I think about it, changing the insert order is still >> sub-optimal as this means that the security policy for the VM will be >> bypassed for those rules. It seems like the security group rules still >> need to be applied on INPUT, not just forward. >> >> On Thu, Jun 18, 2015 at 1:24 PM, Nick Bartos <n...@pistoncloud.com> >> wrote: >> >>> A very recent change modified the input iptables rules. However, this >>> resulted in an unfortunate situation. >>> >>> The felix rules ("INPUT -j felix-INPUT", "FORWARD -j felix-FORWARD") >>> are inserted to the top of the chain, preempting any rules that were set >>> before it. Later in the felix input rule, there is this: >>> >>> -A felix-INPUT -m conntrack --ctstate INVALID -j DROP >>> -A felix-INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT >>> -A felix-INPUT -d 127.0.0.1/32 -i tap+ -p tcp -m tcp --dport 8775 -j >>> ACCEPT >>> -A felix-INPUT -i tap+ -p udp -m udp --sport 68 --dport 67 -j ACCEPT >>> -A felix-INPUT -i tap+ -j DROP >>> >>> This basically means any previous allow rules on the input chain from >>> the VMs are disregarded. This is a problem for us as we run some services >>> locally on the hypervisors that the VMs need to be able to talk to. >>> >>> While this was sort of the case with the forward chain before, it was >>> generally fine, since the outbound security policy was respected (which for >>> most cases is allow). >>> >>> Is there a way to get the felix-INPUT (and maybe the felix-FORWARD) >>> jumps to be inserted further down the list (although putting it way down at >>> the bottom wouldn't work for us as we have a debug log for the catch-all >>> policy drop there)? Or can you think of another solution? >>> >> >> >
_______________________________________________ calico-tech mailing list calico-tech@lists.projectcalico.org http://lists.projectcalico.org/listinfo/calico-tech
