Yep! > That's actually a known bug that has been there "forever". It has always > been brought up in the context of "I want communication between my NATed > networks, and it only works in one direction!", rather than "I want to > block communication between my NATed networks, but it gets past in one > direction!". When people learn that it's (kind of) blocked, they work > around it by adding a new network that is shared by all the guests. > Because all of the reports have been in this direction, fixing it > properly has been very low on (my) priority list.
Thanks for confirming the bug as I suspected, Laine! I knew it was too good to be true that I'd be the first to discover this. Thanks for your huge help, Martin. I tried searching for this age-old you may be speaking of and I was only able to find this: https://bugzilla.redhat.com/show_bug.cgi?id=533193 Also found another somewhat-related bug affecting iptables (port-forwarding to VM): https://bugzilla.redhat.com/show_bug.cgi?id=1079088 If it's not too much trouble, can you provide me the ID number so I can track it? Aside from the extra packet processing overhead caused by the high > number of rules needed (which may be completely insignificant), and the > necessity to add/remove rules at specific labels rather than just > inserting them all at the beginning, there is the issue that currently > each iptables rule added for a libvirt network is dependent only on that > network, so it's simple to determine which rules to remove when a > network is destroyed. If we start adding rules that are dependent on two > networks, then proper cleanup will be more complicated. > None of that should prevent a properly motivated person from writing the > code to do it, of course! At least now I know this is quite feasible to fix. Thank you both for the well-elaborated responses. Cheers, -Travis On Tue, Jun 20, 2017 at 12:46 PM, Laine Stump <[email protected]> wrote: > On 06/20/2017 05:27 AM, Martin Kletzander wrote: > > On Tue, Jun 20, 2017 at 10:05:19AM +0200, Martin Kletzander wrote: > >> On Tue, Jun 20, 2017 at 02:26:59AM -0400, Travis S. Johnson wrote: > >>> Hello, > >>> > >>> I came across an interesting problem in my home lab a few weeks ago > >>> as I'm > >>> prepping for my RHCE exam using Michael Jang study guide. I've been > >>> at this > >>> for days now, and I still can't wrap my head around how two or more > >>> virtual > >>> networks in default NAT configuration are even allowed to communicate > >>> with > >>> each other despite what the libvirt documentation said. > >>> > >>> > >>> Here's the excerpt I'm referring to in the wiki link here: > >>> http://wiki.libvirt.org/page/Networking#Forwarding_ > Incoming_Connections: > >>> > >>>> By default, guests that are connected via a virtual network with > >>>> <forward > >>>> mode='nat'/> can make any outgoing network connection they like. > >>>> Incoming > >>>> connections are allowed from the host, and from other guests > >>>> connected to > >>>> the same libvirt network, but all other incoming connections are > >>>> blocked by > >>>> iptables rules. > >>> > > > > I did not read this properly... > > > >>> > >>> Also here's another assertion from 'The virtual network driver' > >>> section in > >>> http://libvirt.org/firewall.html: > >>> > >>>> type=nat > >>>> > >>>> Allow inbound related to an established connection. Allow outbound, > but > >>>> only from our expected subnet. Allow traffic between guests. Deny > >>>> all other > > > > I would expect the 'traffic between guests' to mean only guests on the > > same network. Also that lines up with what's written above. > > That is correct. > > > > >>>> inbound. Deny all other outbound. > >>> > > > > [...] > > > >> > >> Thanks for reporting this. It's clearly a bug in libvirt. The rules > >> are in this order: > >> > >> all rules for virbr0 > >> all rules for virbr1 > >> all rules for virbr2 > >> > >> But what we should do instead is: > >> > >> input rules for all networks > >> local rules for all networks > >> output rules for all networks > >> reject rules for all networks > > That doesn't achieve the desired results (as you figured out later, > according to your followup comments. > > > >> > >> The problem is that we do not know how other rules look like. So what > >> we might need to do is create chains where rules for the first network > >> are, then only append network rules into those chains. > >> > >> Would you mind filing a bug for this issue, so we can properly track it > >> and don't forget about it? I'll have a look at it in the meantime, but > >> don't promise anything since I'm not that familiar with that part of the > >> codebase. > >> > > > > Actually, now that I'm thinking about it, the problem is not that we > > disallow the communication from virbr1 to virbr0, but the problem is > > that we allow the connection from virbr0 to everywhere. > > Yep! > > That's actually a known bug that has been there "forever". It has always > been brought up in the context of "I want communication between my NATed > networks, and it only works in one direction!", rather than "I want to > block communication between my NATed networks, but it gets past in one > direction!". When people learn that it's (kind of) blocked, they work > around it by adding a new network that is shared by all the guests. > Because all of the reports have been in this direction, fixing it > properly has been very low on (my) priority list. > > > > > > Maybe the solution would be: for each network, insert the rules on top > > of the forward chain and for each started network, explicitly reject > > that one. So that after one network starts it would look like this (I'm > > writing this from memory just to illustrate the idea, not actually > > looking up how stuff looks): > > > > ACCEPT any virbr0 RELATED,ESTABLISHED > > ACCEPT virbr0 virbr0 > > ACCEPT virbr0 any src:192.168.122.0/24 > > REJECT any virbr0 > > REJECT virbr0 any > > > > And after second network is started, we'd have: > > > > # This one is new: > > REJECT virbr1 virbr0 > > Yeah, it's (mostly) this rule that makes the solution complicated. > Essentially you need one of these rules for each pair of virtual > networks, so if you have 10 networks, you'll need 45 rules, for 11 > you'll need 55 (in general, for "n" networks, you'll need (n * (n-1) / > 2) extra rules). > > Aside from the extra packet processing overhead caused by the high > number of rules needed (which may be completely insignificant), and the > necessity to add/remove rules at specific labels rather than just > inserting them all at the beginning, there is the issue that currently > each iptables rule added for a libvirt network is dependent only on that > network, so it's simple to determine which rules to remove when a > network is destroyed. If we start adding rules that are dependent on two > networks, then proper cleanup will be more complicated. > > None of that should prevent a properly motivated person from writing the > code to do it, of course! > > > # These ones would be normally at the end, IIRC: > > ACCEPT any virbr1 RELATED,ESTABLISHED > > ACCEPT virbr1 virbr1 > > ACCEPT virbr1 any src:192.168.122.0/24 > > REJECT any virbr1 > > REJECT virbr1 any > > # These are left as they were: > > ACCEPT any virbr0 RELATED,ESTABLISHED > > ACCEPT virbr0 virbr0 > > ACCEPT virbr0 any src:192.168.122.0/24 > > REJECT any virbr0 > > REJECT virbr0 any > > > > Cc'ing Laine so that he can weigh in as he has way more knowledge of > > this part of the code =) >
_______________________________________________ libvirt-users mailing list [email protected] https://www.redhat.com/mailman/listinfo/libvirt-users
