Correction, when I switch from -g to -m, external works, internal fails (lack
of sleep from setting up environment all night :)).
________________________________
From: [EMAIL PROTECTED] on behalf of Gary W. Smith
Sent: Thu 9/13/2007 8:03 AM
To: LinuxVirtualServer.org users mailing list.; LinuxVirtualServer.org users
mailing list.
Subject: Re: [lvs-users] IPVSADM/IPTables question
Joe,
I have included a configuration from my test environment below. I've had mixes
success and am probably missing something simple. I can reliably hit the LVS
machines from anything inside the test network, but anything outside the
firewall box (which is currently wide open) doesn't work. Below are all of th
configuration's including actual IP's.
I'm unsure if I'm running the routes right. One thing I read said delete the
local routes and to force everything out the firewall. Somewhere I read that I
should use ipip (someones are all google notes, with no response to if it
worked or not).
I've also tried using -m instead of -g, but that didn't seem to make a
difference.
Can you take a look at the config and let me know if something stands out as
broken?
Thanks,
Gary
Test configuration:
4 boxes, configured as follows:
Box 1, firewall/director.
[EMAIL PROTECTED] sysconfig]# ip addr
1: lo: <LOOPBACK,UP,10000> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:40:05:86:c8:7f brd ff:ff:ff:ff:ff:ff
inet 10.10.17.254/24 brd 10.10.17.255 scope global eth0
inet6 fe80::240:5ff:fe86:c87f/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:02:a5:59:42:b2 brd ff:ff:ff:ff:ff:ff
inet 10.0.16.44/24 brd 10.0.16.255 scope global eth1
inet 10.0.16.45/24 scope global secondary eth1
inet6 fe80::202:a5ff:fe59:42b2/64 scope link
valid_lft forever preferred_lft forever
[EMAIL PROTECTED] sysconfig]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.0.16.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
10.10.17.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
0.0.0.0 10.0.16.254 0.0.0.0 UG 0 0 0 eth1
[EMAIL PROTECTED] sysconfig]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.16.45:http wlc
-> 10.10.17.6:http Route 100 0 0
-> 10.10.17.5:http Route 100 0 0
[EMAIL PROTECTED] sysconfig]# iptables-save
# Generated by iptables-save v1.3.1 on Thu Sep 13 06:29:46 2007
*nat
:PREROUTING ACCEPT [1302:307491]
:POSTROUTING ACCEPT [6:324]
:OUTPUT ACCEPT [32:1946]
-A PREROUTING -i eth1 -p tcp -m tcp --dport 20005 -j DNAT --to-destination
10.10.17.5:22
-A PREROUTING -i eth1 -p tcp -m tcp --dport 20006 -j DNAT --to-destination
10.10.17.6:22
-A PREROUTING -i eth1 -p tcp -m tcp --dport 20007 -j DNAT --to-destination
10.10.17.7:22
-A POSTROUTING -o eth1 -j MASQUERADE
COMMIT
# Completed on Thu Sep 13 06:29:46 2007
[EMAIL PROTECTED] sysconfig]# grep forward /etc/sysctl.conf
# Controls IP packet forwarding
net.ipv4.ip_forward = 1
Box 2, test web server A
[EMAIL PROTECTED] ~]# ip addr
1: lo: <LOOPBACK,UP,10000> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet 10.0.16.45/32 scope global lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0d:88:3d:c5:82 brd ff:ff:ff:ff:ff:ff
inet 10.10.17.5/24 brd 10.10.17.255 scope global eth0
inet6 fe80::20d:88ff:fe3d:c582/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 00:02:a5:03:de:d5 brd ff:ff:ff:ff:ff:ff
[EMAIL PROTECTED] ~]# grep arp /etc/sysctl.conf
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.eth0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
[EMAIL PROTECTED] ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.10.17.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 10.10.17.254 0.0.0.0 UG 0 0 0 eth0
Box 3, test web server B
[EMAIL PROTECTED] ~]# ip addr
1: lo: <LOOPBACK,UP,10000> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet 10.0.16.45/32 scope global lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:40:05:7d:98:ca brd ff:ff:ff:ff:ff:ff
inet 10.10.17.6/24 brd 10.10.17.255 scope global eth0
inet6 fe80::240:5ff:fe7d:98ca/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 00:02:a5:0f:64:54 brd ff:ff:ff:ff:ff:ff
[EMAIL PROTECTED] ~]# grep arp /etc/sysctl.conf
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.eth0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
[EMAIL PROTECTED] ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.10.17.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 10.10.17.254 0.0.0.0 UG 0 0 0 eth0
Box 4, test box on the inside of the dev firewall
[EMAIL PROTECTED] ~]# ip addr
1: lo: <LOOPBACK,UP,10000> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0d:88:46:1b:22 brd ff:ff:ff:ff:ff:ff
inet 10.10.17.7/24 brd 10.10.17.255 scope global eth0
inet6 fe80::20d:88ff:fe46:1b22/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 00:02:a5:1f:85:ad brd ff:ff:ff:ff:ff:ff
[EMAIL PROTECTED] ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.10.17.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 10.10.17.254 0.0.0.0 UG 0 0 0 eth0
[EMAIL PROTECTED] ~]# wget 10.0.16.44
--04:02:54-- http://10.0.16.44/ <http://10.0.16.44/>
=> `index.html'
Connecting to 10.0.16.44:80... failed: Connection refused.
[EMAIL PROTECTED] ~]# wget 10.0.16.45 <== Expected response, HTTP answered
--04:02:56-- http://10.0.16.45/ <http://10.0.16.45/>
=> `index.html'
Connecting to 10.0.16.45:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
04:02:57 ERROR 403: Forbidden.
Box 4, text box on the outside of the dev firewall
[EMAIL PROTECTED] ~]# ip addr
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:16:ff:aa:00:01 brd ff:ff:ff:ff:ff:ff
inet 10.0.16.171/24 brd 10.0.16.255 scope global eth0
inet6 fe80::216:ffff:feaa:1/64 scope link
valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480 qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0
[EMAIL PROTECTED] ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.0.16.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 10.0.16.254 0.0.0.0 UG 0 0 0 eth0
[EMAIL PROTECTED] ~]# ping 10.0.16.45 -c 1
PING 10.0.16.45 (10.0.16.45) 56(84) bytes of data.
64 bytes from 10.0.16.45: icmp_seq=0 ttl=64 time=0.281 ms
--- 10.0.16.45 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.281/0.281/0.281/0.000 ms, pipe 2
[EMAIL PROTECTED] ~]# telnet 10.0.16.45 80
Trying 10.0.16.45...
telnet: connect to address 10.0.16.45: Connection timed out
_______________________________________________
LinuxVirtualServer.org mailing list - [email protected]
Send requests to [EMAIL PROTECTED]
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
_______________________________________________
LinuxVirtualServer.org mailing list - [email protected]
Send requests to [EMAIL PROTECTED]
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
