On 04/02/2013 03:31 PM, Laine Stump wrote:
On 03/15/2013 02:10 PM, Gene Czarcinski wrote:
This patch adds support for adding a static route for
a network. The "via" specifies the gateway's IP
address. Both IPv4 and IPv6 static routes are
supported although it is expected that this
functionality will have more use with IPv6.
(First I want to make sure I correctly understand what you're wanting to
do, so I'm going to try and explain it in my own words...)
>From reading your earlier messages, my understanding is that the aim of
this patch is to automatically setup a route to a virtual network whose
bridge device has no IP address assigned, and is therefore reachable
only via one of the guests, is this correct?
In other words, let's say that I have the following two networks defined
(using IPv4 and all static IPs for brevity, but the entire discussion is
equally applicable to IPv6):
<network>
<name>netVisibleToHost</name>
<bridge name='virbr1'/>
<forward mode='route'/>
<ip address='192.168.200.1' prefix='24'/>
</network>
<network>
<name>netHiddenFromHost</name>
<bridge name='virbr2'/>
</network>
and you have a guestDirect that has two interfaces:
<interface type='network'> <!-- 192.168.200.2/24 -->
<source network='netVisibleToHost'/>
</interface>
<interface type='network'> <!-- 192.168.201.1/24 -->
<source network='netHiddenFromHost'/>
</interface>
and another guestIndirect that has only one interface:
<interface type='network'> <!-- 192.168.201.2/24 -->
<source network='netHiddenFromHost'/>
</interface>
Additionally, the default route on guestDirect points to 192.168.200.1,
and the default route on guestIndirect points to 192.168.201.1.
(Presumably you don't want to simply add an IP address to
netHiddenFromHost because (while it would solve your routing problems)
it would violate some security policy you've built into your network
topology - namely that all traffic to and from netHiddenFromHost *must*
go through guestDirect.)
Traffic outbound from guestIndirect would have no problem reaching its
destination - it would go across netHiddenFromHost to guestDirect
(192.168.201.1), which would know to forward it to the host
(192.168.200.1) via netVisibleToHost, and the host presumably knows how
to get to anywhere. The problem comes when trying to route the
*response* destined for 192.168.201.2 (guestIndirect) - the outside
world may know that those packets have to be sent to the host, but the
host doesn't have a route for 192.168.201.0/24 - only guestDirect does.
So, the solution is to add a route on the *host* that points traffic
destined for 192.168.201.0/24 to guestDirect, a.k.a. 192.168.200.2.
Since there's no place in /etc/sysconfig/network-scripts/route-* to put
static routes with destinations that are only reachable through a
transient interface such as the bridge devices created by libvirt, the
obvious solution is to cause libvirt to add such a route, and the way
you propose to do that is to add an <ip> element in the network named
"netUnreachable".
Am I understanding the issue so far?
I believe you do understand.
I have not read the rest of your message yet but I did want to bring up
one point.
With IPv4 I can get a lot of this functionality by the cleaver use of
NAT but with IPv6 there is no NAT and everything is routed. The big
issue with IPv6 (and the reason I did this) is to provide the
virtualization host with information needed to route response packets
back to the "hidden" guest. Without this, the can can send any packet
it wants but will never get a reply. That is the basic problem being
addressed.
Assuming that I am, then as far as I can see, the correct place to
configure this route isn't in the setup for netHiddenFromHost, but
rather in netVisibleToHost - this is more in line with the way static
routes are configured in the standard host network config (route-*
files), and eliminates the problem that would occur if netHiddenFromHost
was started before netVisibleToHost existed (the route would be added
pointing at the wrong interface, if at all)
You are correct. The "route-via" must be defined in the "visible"
network definition. There is an additional constraint that the gateway
address must be in the range of the visible network. IIRC, tere are a
couple examples in the code I added for tests.
So, what I'm proposing is that, to automatically setup the route in the
above example, netHiddenFromHost would remain unchanged, and
netVisibleToHost would look something like this:
<network>
<name>netVisibleToHost</name>
<bridge name='virbr1'/>
<forward mode='route'/>
<ip address='192.168.200.1' prefix='24'/>
<route address='192.168.201.0' prefix='24' gateway='192.168.200.2'/>
</network>
The route element could also have a family attribute just as <ip> does
(although it's fairly simple to figure out the family just by trying to
parse address and/or gateway). You might instead want to make <route> a
subelement of <ip>, then validate that the gateway address is directly
reachable from the given ip address (i.e. that it's on the same subnet).
The <route> above is similar to what I implemented. Since I am defining
a network which is "somehow" reachable via this network interface, I
used the "regular" <ip> element to define the network and then added the
"via=" (instead of "gateway="). One reason I chose to use "via=" rather
than "gateway=" is that when you run the
ip -6 route or ip -4 route
you get:
<ip-address> via <gateway-address>
Here is a part of a definition I have:
<network>
<name>net7</name>
...
<ip family='ipv6' address='fd00:beef:10:7::1' prefix='64'>
<dhcp>
<range start='fd00:beef:10:7::1:1' end='fd00:beef:10:7::1:f1' />
</dhcp>
</ip>
<ip family='ipv6' address='fd00:beef:12::' prefix='48'
via='fd00:beef:10:7::7:2'>
</ip>
...
</network>
and here is what you get from ip -6 route:
fd00:beef:10:7::/64 dev virbr3 proto kernel metric 256
fd00:beef:12::/48 via fd00:beef:10:7::7:2 dev virbr3 proto static metric 1
By putting the configuration here, you could be assured that the
interface that will be used for the route will always exist at the time
the route is added. Also it is conceptually more similar to the way that
the routes in /etc/sysconfig/route-ethX all have gateways that are
directly reachable via "ethX".
Absolutely. "Floating" routes make no sense to me. This static route
is closely coupled with the (main) address on that virtual network
definition and needs to be included in the definition.
*OR* (following is what I think is a bad idea, but maybe someone can
tweak and salvage it :-)
I cringe here. Since what you stated above is more or less what I
implemented, I am of the opinion that this is the one that makes sense.
If you use NetworkManager to define some static routes, those
definitions will need to be on the which of the network where the
gateway is reachable. Mmmm .. I just noticed that NetworkManager used
"Gateway" and not "via" ... maybe I should change.
Here is an alternate proposal that has the advantage of tying the
existence of the static route to the existence of not just the bridge,
but of even the guest interface will be the gateway: instead of putting
the route in the configuration of the netVisibleToHost network, put it
directly in the configuration of the guest interface itself. That way
when the guest is started the route will be added, and when the guest is
shutdown, the route (which will anyway now be useless) will be removed.
That could be added to the <interface> definition something like this:
<interface type='network'> <!-- 192.168.200.2/24 -->
<source network='netVisibleToHost'/>
<route address='192.168.201.0' prefix='24' gateway='192.168.200.2'/>
</interface>
If this was done correctly, you could even hotplug a guest interface
that would then be immediately used for a route, as well as updating an
existing guest interface to add a route. The route added would use the
information in the <route> element plus the bridge device the guest's
tap device was connected to.
I do have several reservations about this idea, though
1) Up until now there has been no "pollution" of the guest config with
IP-related info, it has all been about hardware. (well, except for
filterref parameters...)
2) Even worse than that is that the guest interface config *can't* know
the IP address that the guest will use for the interface, yet we would
now be hard-coding it into this new <route> element in the guest
interface config. Ugh. (Now what would be *really* slick is if we could
have routes in <interface> that learned their IP address from nwfilter's
IP address snooping! :-)
3) If the interface happens to be connected to a bridge that has no IP
address on the host (e.g. the bridge of netHiddenFromHost in the example
above), the route will be a failure.
4) Likewise, if the interface is connected with macvtap, again the route
would be a failure.
So, in the end, the idea of adding <route> to <interface> is probably a
no-go.
Yes, nogo with me also.
In many respects the route definition is not much different than other
network definition included in a virtual network definition except that
it is only for routing.
(One upside - if the interface is a hostdev, we could simply add the
route with no "dev X" appended, and it *would* work, as long as the host
had an interface on the same subnet).
The command used is of the following form:
ip route add <address>/<prefix> via <gateway> dev <virbr-bridge> \
proto static metric 1
It seems to me that the "dev" given here is incorrect. If I'm not
mistaken, the route you're adding with your current code will use the
device name of the bridge on "netHiddenFromHost" in this command, while
what you *really* want is the bridge used for "netVisibleToHost" (which
isn't even known/available at the time you start netHiddenFromHost -
another reason to do it in one of the ways I'm suggesting).
No, no. The <virbr-bridge> is defininiton the visible virtual network
interface. The example I cited about is currently "running" with
libvirt which includes the patch. Here is the result for ip -6 addr:
3: virbr3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500
inet6 fd00:beef:10:7::1/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fef8:9ca7/64 scope link
valid_lft forever preferred_lft forever
So, gateway fd00:beef:10:7::7:2 is covered by network
fd00:beef:10:7::/64 which is on device virbr3. The network
fd00:beef:12::/48 is routed via fd00:beef:10:7::7:2 on virbr3.
I do not know if you have given these patches a try but it does make
more sense when you see it work.
BTW, one static route for IPv4 and one static route for IPv6 is
supported by my "IPv6" updates to virt-manager. I need to do a little
more work to add that to the virtual-network-creation-wizard.
Gene
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
