Re: [Vserver] advanced routing per vps
On Wed, Sep 20, 2006 at 03:30:08PM +0200, Rik Bobbaers wrote: > Herbert Poetzl wrote: > >routes and addresses are only loosely coupled, > >and the existance of one doesn't imply the other > > not really... i can do a: > ip route add $NETWORK dev $IF table > > but, i can't add a gateway for that network if there is > no ip defined on that $IF why would you want to add a gateway route without having an ip on the same network? (i.e. that doesn't make sense IMHO) > so, how would i go around that? > > >yes, definitely, I'd put one routing table > >per vserver guest, this way you can basically > >have per guest routing and it will not affect > >the other guests at all ... > > wouldn't that be a lot of overhead? i'd say just add rules > in start and stop scripts, make the routes for each table > fixed at boottime or so... yes, but boottime of the guest (if you want per guest routing) > (that was my initial thought, but as you see above, you > can't add a gateway to a route if you don't have an ip > on that network... I just assume that all guests will have an ip on the network they are trying to route over :) > sollutio would indeed be: put a route for every vps on > your system... a separate routing entry (in a separate table) for each vps, yes, that's what you want if you have per guest/vps routing needs ... > (requires more maintenance on your rt_tables file tough...) well, only if you want named tables, they work pretty fine with numebrs too :) > but might be a good idea... i don't know yet :) keep us posted, best, Herbert > -- > harry > aka Rik Bobbaers > > K.U.Leuven - LUDIT -=- Tel: +32 485 52 71 50 > [EMAIL PROTECTED] -=- http://harry.ulyssis.org > > "Work hard and do your best, it'll make it easier for the rest" > -- Garfield > > Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm > > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] advanced routing per vps
Herbert Poetzl wrote: routes and addresses are only loosely coupled, and the existance of one doesn't imply the other not really... i can do a: ip route add $NETWORK dev $IF table but, i can't add a gateway for that network if there is no ip defined on that $IF so, how would i go around that? yes, definitely, I'd put one routing table per vserver guest, this way you can basically have per guest routing and it will not affect the other guests at all ... wouldn't that be a lot of overhead? i'd say just add rules in start and stop scripts, make the routes for each table fixed at boottime or so... (that was my initial thought, but as you see above, you can't add a gateway to a route if you don't have an ip on that network... sollutio would indeed be: put a route for every vps on your system... (requires more maintenance on your rt_tables file tough...) but might be a good idea... i don't know yet :) -- harry aka Rik Bobbaers K.U.Leuven - LUDIT -=- Tel: +32 485 52 71 50 [EMAIL PROTECTED] -=- http://harry.ulyssis.org "Work hard and do your best, it'll make it easier for the rest" -- Garfield Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] advanced routing per vps
On Mon, Sep 18, 2006 at 04:02:27PM +0200, Rik Bobbaers wrote: > Herbert Poetzl wrote: > > >you do not need an ip address to bring an interface up :) > > > > ifconfig eth2 up > > ifconfig eth2 down > > as said... if i restart interfaces, i don't want "weird old" ip > addresses on the interface, so i put 0.0.0.0 on it ;) > not really sure if that makes sense, but... > > don't even know if that's a good idea... anyone comments? > > >this config option will become a 'vlandev' in the near > >future (probably already is in CVS/SVN), which will > >make more sense here ... > > mkay, tnx :) > > >use counts are generally a bad idea, as we already > >saw with the mainline behaviour on removing the > >primary ip, etc ... mainly because you would have > >to account for host actions too ... but feel free > >to write your own 'book keeping' scripts and hook > >them into the startup/shutdown > > yeah, you don't want a real use count, but you COULD check if there > are routes over that vlan, if there are still routes, that means, > there are still ip's on that vlan, so don't remove. or am i wrong > here? routes and addresses are only loosely coupled, and the existance of one doesn't imply the other > >wouldn't it be more appropriate to add those routes > >to the appropriate tables? > > how do you mean? i don't think i completely get it... you see the > scripts, what should be changed??? right now, i have 1 table per > configured vlan. you suggest 1 table per virtual server? yes, definitely, I'd put one routing table per vserver guest, this way you can basically have per guest routing and it will not affect the other guests at all ... HTH, Herbert > greetz, > > -- > harry > aka Rik Bobbaers > > K.U.Leuven - LUDIT -=- Tel: +32 485 52 71 50 > [EMAIL PROTECTED] -=- http://harry.ulyssis.org > > "Work hard and do your best, it'll make it easier for the rest" > -- Garfield > > Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm > > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] advanced routing per vps
Herbert Poetzl wrote: you do not need an ip address to bring an interface up :) ifconfig eth2 up ifconfig eth2 down as said... if i restart interfaces, i don't want "weird old" ip addresses on the interface, so i put 0.0.0.0 on it ;) not really sure if that makes sense, but... don't even know if that's a good idea... anyone comments? this config option will become a 'vlandev' in the near future (probably already is in CVS/SVN), which will make more sense here ... mkay, tnx :) use counts are generally a bad idea, as we already saw with the mainline behaviour on removing the primary ip, etc ... mainly because you would have to account for host actions too ... but feel free to write your own 'book keeping' scripts and hook them into the startup/shutdown yeah, you don't want a real use count, but you COULD check if there are routes over that vlan, if there are still routes, that means, there are still ip's on that vlan, so don't remove. or am i wrong here? wouldn't it be more appropriate to add those routes to the appropriate tables? how do you mean? i don't think i completely get it... you see the scripts, what should be changed??? right now, i have 1 table per configured vlan. you suggest 1 table per virtual server? greetz, -- harry aka Rik Bobbaers K.U.Leuven - LUDIT -=- Tel: +32 485 52 71 50 [EMAIL PROTECTED] -=- http://harry.ulyssis.org "Work hard and do your best, it'll make it easier for the rest" -- Garfield Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] advanced routing per vps
On Fri, Sep 15, 2006 at 11:35:26AM +0200, Rik Bobbaers wrote:
> heya all,
>
> since yesterday , i found it necessary to do my routing per vserver.
> so what did i do: (i put everything in /usr/local btw ;))
>
> gandalf:~# cat /usr/local/etc/vservers/.defaults/scripts/pre-start
> #!/bin/sh
>
> HOSTNAME=$2
>
> IF=`cat /usr/local/etc/vservers/${HOSTNAME}/interfaces/0/dev`
> IP=`cat /usr/local/etc/vservers/${HOSTNAME}/interfaces/0/ip`
> NETMASK=`cat /usr/local/etc/vservers/${HOSTNAME}/interfaces/0/prefix`
> # this is the case on all our networks... might not work for you ;)
> GW=`ipcalc -n $IP $NETMASK |grep "HostMax:"| awk '{print $2}'`
> NETWORK=`ipcalc -n $IP $NETMASK |grep "Network:"| awk '{print $2}'`
>
> ip route add $NETWORK dev $IF table $IF-net
> ip route add default via $GW dev $IF table $IF-net
> ip rule add from $IP/32 table $IF-net pref 1000
> EOF
>
> gandalf:~# cat /usr/local/etc/vservers/.defaults/scripts/post-stop
> #!/bin/sh
>
> HOSTNAME=$2
>
> IF=`cat /usr/local/etc/vservers/${HOSTNAME}/interfaces/0/dev`
> IP=`cat /usr/local/etc/vservers/${HOSTNAME}/interfaces/0/ip`
>
> ip rule del from $IP/32 table $IF-net pref 1000
> EOF
>
> now... i NEED to have all the interfaces in /etc/iproute2/rt_tables to
> make this work (no problem, a routing table per VLAN suits me fine ;))
>
> i also have to make sure the interfaces are all up @ boot (no problem,
> since:
> gandalf:~# cat /etc/network/interfaces
> # /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
>
> auto lo
> iface lo inet loopback
>
> auto eth0.49
> iface eth0.49 inet manual
> pre-up ifconfig eth0 hw ether 00:15:BA:DC:0D:ED
> up ifconfig eth0.49 0.0.0.0 up
> auto eth0.164
> iface eth0.164 inet manual
> pre-up ifconfig eth0 hw ether 00:15:BA:DC:0D:ED
> up ifconfig eth0.164 0.0.0.0 up
>
> auto eth1
> iface eth1 inet static
> address 192.168.28.30
> netmask 255.255.254.0
> broadcast 192.168.29.255
> gateway 192.168.29.254
> post-up route add -net 192.168.30.0 netmask 255.255.255.0 gw
> 192.168.29.253
> pre-down route del -net 192.168.30.0 netmask 255.255.255.0 gw
> 192.168.29.253
>
> auto eth2.94
> iface eth2.94 inet static
> address 134.58.241.34
> netmask 255.255.255.0
> broadcast 134.58.241.255
> auto eth2.95
> iface eth2.95 inet manual
> up ifconfig eth2.95 0.0.0.0 up
>
> (the static ip on eth2.94 is for allowing nfs in a vps... it seems
> impossible to make the "source address" for nfs the same as the vps
> address... but that's another problem ;))
>
> i use vlans, so i have to set every vlan UP @ boottime and all guests
> novlandev.
>
> now my questions:
> 1. the device i have to use for my hosts is: eth2.94, so i put that in
> dev. if i boot my machine, i don't have eth2 up, because the host itself
> doesn't need to have an ip address on that network. if i want to start a
> vps on eth2.94, and let vserver create the vlan for me, it doesn't
> work if eth2 isn't up... why is that? can't vserver check if eth2 is up,
> and if it's not, then set it up? what's the reason for that?
>
> sollution: bring up eth2 at boottime without an address and all works
> fine. but this gets us to the next problem...
you do not need an ip address to bring an interface up :)
ifconfig eth2 up
ifconfig eth2 down
> 2. the vps sets up the vlan nicely, sets up networking, scripts make
> sure routing is done fine. but when i set up 2 hosts on the same vlan,
> and i shut down 1 of those vps'es, it REMOVES the vlan dev, and the
> other vps lose their network, while it's still in use!!!
>
> sollution: put a novlandev in each hosts config. BUT (here we go again)
> that means i have to do the vlan config myself again. (not that it's a
> problem, since you can see my interfaces file now).
this config option will become a 'vlandev' in the near
future (probably already is in CVS/SVN), which will
make more sense here ...
> 3. how hard is it to implement a "use count" or so for those kind of
> things? just check if there are other vps'es using the vlan dev. if
> noone uses it, THEN bring it down. if there are vps'es using the vlan:
> leave it alone. that way, the last vps using the vlan will disable it,
> the first one needing it, will start it.
use counts are generally a bad idea, as we already
saw with the mainline behaviour on removing the
primary ip, etc ... mainly because you would have
to account for host actions too ... but feel free
to write your own 'book keeping' scripts and hook
them into the startup/shutdown
> for ip addresses, we have net.ipv4.conf.all.promote_secondaries=1
> something alike for vlan devs would be nice ;)
>
> 4. totally different now... the way i start my advanced routing, it
> allways says (except the first time off course):
> RTNETLINK answers: File exists
> RTNETLINK answers: File exists
> (normal, the route to that network exists)
> That's the reason i can't del
[Vserver] advanced routing per vps
heya all,
since yesterday , i found it necessary to do my routing per vserver.
so what did i do: (i put everything in /usr/local btw ;))
gandalf:~# cat /usr/local/etc/vservers/.defaults/scripts/pre-start
#!/bin/sh
HOSTNAME=$2
IF=`cat /usr/local/etc/vservers/${HOSTNAME}/interfaces/0/dev`
IP=`cat /usr/local/etc/vservers/${HOSTNAME}/interfaces/0/ip`
NETMASK=`cat /usr/local/etc/vservers/${HOSTNAME}/interfaces/0/prefix`
# this is the case on all our networks... might not work for you ;)
GW=`ipcalc -n $IP $NETMASK |grep "HostMax:"| awk '{print $2}'`
NETWORK=`ipcalc -n $IP $NETMASK |grep "Network:"| awk '{print $2}'`
ip route add $NETWORK dev $IF table $IF-net
ip route add default via $GW dev $IF table $IF-net
ip rule add from $IP/32 table $IF-net pref 1000
EOF
gandalf:~# cat /usr/local/etc/vservers/.defaults/scripts/post-stop
#!/bin/sh
HOSTNAME=$2
IF=`cat /usr/local/etc/vservers/${HOSTNAME}/interfaces/0/dev`
IP=`cat /usr/local/etc/vservers/${HOSTNAME}/interfaces/0/ip`
ip rule del from $IP/32 table $IF-net pref 1000
EOF
now... i NEED to have all the interfaces in /etc/iproute2/rt_tables to
make this work (no problem, a routing table per VLAN suits me fine ;))
i also have to make sure the interfaces are all up @ boot (no problem,
since:
gandalf:~# cat /etc/network/interfaces
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
auto lo
iface lo inet loopback
auto eth0.49
iface eth0.49 inet manual
pre-up ifconfig eth0 hw ether 00:15:BA:DC:0D:ED
up ifconfig eth0.49 0.0.0.0 up
auto eth0.164
iface eth0.164 inet manual
pre-up ifconfig eth0 hw ether 00:15:BA:DC:0D:ED
up ifconfig eth0.164 0.0.0.0 up
auto eth1
iface eth1 inet static
address 192.168.28.30
netmask 255.255.254.0
broadcast 192.168.29.255
gateway 192.168.29.254
post-up route add -net 192.168.30.0 netmask 255.255.255.0 gw
192.168.29.253
pre-down route del -net 192.168.30.0 netmask 255.255.255.0 gw
192.168.29.253
auto eth2.94
iface eth2.94 inet static
address 134.58.241.34
netmask 255.255.255.0
broadcast 134.58.241.255
auto eth2.95
iface eth2.95 inet manual
up ifconfig eth2.95 0.0.0.0 up
(the static ip on eth2.94 is for allowing nfs in a vps... it seems
impossible to make the "source address" for nfs the same as the vps
address... but that's another problem ;))
i use vlans, so i have to set every vlan UP @ boottime and all guests
novlandev.
now my questions:
1. the device i have to use for my hosts is: eth2.94, so i put that in
dev. if i boot my machine, i don't have eth2 up, because the host itself
doesn't need to have an ip address on that network. if i want to start a
vps on eth2.94, and let vserver create the vlan for me, it doesn't
work if eth2 isn't up... why is that? can't vserver check if eth2 is up,
and if it's not, then set it up? what's the reason for that?
sollution: bring up eth2 at boottime without an address and all works
fine. but this gets us to the next problem...
2. the vps sets up the vlan nicely, sets up networking, scripts make
sure routing is done fine. but when i set up 2 hosts on the same vlan,
and i shut down 1 of those vps'es, it REMOVES the vlan dev, and the
other vps lose their network, while it's still in use!!!
sollution: put a novlandev in each hosts config. BUT (here we go again)
that means i have to do the vlan config myself again. (not that it's a
problem, since you can see my interfaces file now).
3. how hard is it to implement a "use count" or so for those kind of
things? just check if there are other vps'es using the vlan dev. if
noone uses it, THEN bring it down. if there are vps'es using the vlan:
leave it alone. that way, the last vps using the vlan will disable it,
the first one needing it, will start it.
for ip addresses, we have net.ipv4.conf.all.promote_secondaries=1
something alike for vlan devs would be nice ;)
4. totally different now... the way i start my advanced routing, it
allways says (except the first time off course):
RTNETLINK answers: File exists
RTNETLINK answers: File exists
(normal, the route to that network exists)
That's the reason i can't delete the routing for the network when i stop
the vps... others may still need it. and if it allready exists, it's ok
by me, so that "error" is just fine.
does anyone have a clean sollution for that?
that's about it for the moment, i think...
just to be clear: it all works fine now... so there is not really a
problem, just some practical questions!
greetz,
--
harry
aka Rik Bobbaers
K.U.Leuven - LUDIT -=- Tel: +32 485 52 71 50
[EMAIL PROTECTED] -=- http://harry.ulyssis.org
"Work hard and do your best, it'll make it easier for the rest"
-- Garfield
Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
___
Vserver mailing list
Vserver@list.linux-vserver.org
http://list.linux-vserver.org/mailman/listinfo/vserver
