Hi Partica, Sorry, I haven't read all your mail, but I think it's may helpful to you: http://www.austintek.com/LVS/LVS-HOWTO/mini-HOWTO/LVS-mini-HOWTO.html#forwarding And I think RHEL is good enough for LVS.
Goodluck, -giobuon On Fri, Oct 23, 2009 at 2:50 AM, Partica Cristian <[email protected]>wrote: > Wha't i'm trying to do is setup a load banancer using 3 servers for a > webservice (HTTP) > > 1st - the main server which will redirect to the 2nd or 3rd server. I > called > this LVS (and it's where ldirectord is) > > 2nd - runs a simple http service and when a client enters...and puts the > ip, > or domain of the 1st server, he should get the contents from the 2nd server > (if up). eighterwise the content from the 3rd server > > 3rd - runs the same http service.. > > the only thing different from the ALL tutorials that i've read is that > those > 3 servers are on the internet , with public ips not behind some private > 192.168.xx network > > 2 of the ip's are in the same subbet of a class C > 1 is not even in the same continent :) > > So a relatively simple thing to do , that ldirectord doesn't do..in this > case > ---------------------- > > I have 3 PUBLIC ip's like (none are with 192..) , two of them are on the > same subnet, > I'm running Cent-oS 5, and here is a COMPLETE list of what i have done. > > XX.XX.XX.234 (this is the lvs..) > XX.XX.XX.235 real server (web ngix) > YYY.YYY.YYY.163 real server (web apache) - I don't really care about this > one, i can move it into the > > we will use XX.XX.XX.236 as virtual..(there isn't an ip on the net with > that > number up). of course it will be simpler to use a 192.. but, i have tried > that also ,and no luck > > same subnet as the first 2 ones, i just want to make it work from the 234 > -> > 235..but it gives me a timeout on the browser... > Here's what i did: > > [r...@linux ~]# cat /etc/ha.d/ldirectord.cf > checktimeout=3 > checkinterval=10 > autoreload=yes > logfile="/var/log/ldirectord.log" > quiescent=no > virtual=XX.XX.XX.236:80 > fallback=127.0.0.1:80 > real=XX.XX.XX.235:80 gate > real=YYY.YYY.YYY.235:80 gate > service=http > request="test.html" > receive="Still alive" > scheduler=rr #here i've tried with wlr as well > protocol=tcp > checktype=negotiate > > [r...@linux ~]# /usr/sbin/ldirectord -d /etc/ha.d/ldirectord.cf start > > DEBUG2: Running exec(/usr/sbin/ldirectord -d /etc/ha.d/ldirectord.cfstart) > Running exec(/usr/sbin/ldirectord -d /etc/ha.d/ldirectord.cf start) > DEBUG2: Invoking ldirectord invoked as: /usr/sbin/ldirectord /etc/ha.d/ > ldirectord.cf start > Invoking ldirectord invoked as: /usr/sbin/ldirectord > /etc/ha.d/ldirectord.cfstart > DEBUG2: Starting Linux Director v1.186-ha-2.1.4 with pid: 18619 > Starting Linux Director v1.186-ha-2.1.4 with pid: 18619 > DEBUG2: Running system(/sbin/ipvsadm -A -t XX.XX.XX.236:80 -s rr ) > Running system(/sbin/ipvsadm -A -t XX.XX.XX.236:80 -s rr ) > DEBUG2: Added virtual server: XX.XX.XX.236:80 > Added virtual server: XX.XX.XX.236:80 > DEBUG2: Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r > 127.0.0.1:80-g -w 1) > Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r 127.0.0.1:80 -g -w > 1) > DEBUG2: Added fallback server: 127.0.0.1:80 (XX.XX.XX.236:80) (Weight set > to > 1) > Added fallback server: 127.0.0.1:80 (XX.XX.XX.236:80) (Weight set to 1) > DEBUG2: Disabled real > server=negotiate:http:tcp:XX.XX.XX.235:80:::1:gate:\/test\.html:Still\ > alive > (virtual=tcp:XX.XX.XX.236:80) > DEBUG2: Checking negotiate: real > server=negotiate:http:tcp:XX.XX.XX.235:80:::1:gate:\/test\.html:Still\ > alive > (virtual=tcp:XX.XX.XX.236:80) > DEBUG2: check_http: url="http://XX.XX.XX.235:80/test.html"; > virtualhost="XX.XX.XX.235" > LWP::UserAgent::new: () > DEBUG2: Starting Check > DEBUG2: Starting HTTP/HTTPS > LWP::UserAgent::request: () > LWP::UserAgent::send_request: GET > http://XX.XX.XX.235:80/test.html<http://xx.xx.xx.235/test.html> > LWP::UserAgent::_need_proxy: Not proxied > LWP::Protocol::http::request: () > LWP::Protocol::collect: read 12 bytes > LWP::UserAgent::request: Simple response: OK > DEBUG2: Finished HTTP/HTTPS > DEBUG2: Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r > XX.XX.XX.235:80 -g -w 1) > Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w > 1) > DEBUG2: system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w > 1) failed: No child processes > system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w 1) > failed: No child processes > DEBUG2: Added real server: XX.XX.XX.235:80 (XX.XX.XX.236:80) (Weight set to > 1) > Added real server: XX.XX.XX.235:80 (XX.XX.XX.236:80) (Weight set to 1) > DEBUG2: Running system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80 > ) > Running system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80) > DEBUG2: system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80) > failed: > No child processes > system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80) failed: No > child > processes > DEBUG2: Deleted fallback server: 127.0.0.1:80 (XX.XX.XX.236:80) > Deleted fallback server: 127.0.0.1:80 (XX.XX.XX.236:80) > DEBUG2: Enabled real > server=negotiate:http:tcp:XX.XX.XX.235:80:::1:gate:\/test\.html:Still\ > alive > (virtual=tcp:XX.XX.XX.236:80) > DEBUG2: check_http: > http://XX.XX.XX.235:80/test.html<http://xx.xx.xx.235/test.html>is up > > If i go to the webserver i can see that ldirector is actually testing the > test.html..every 10 seconds like in the conf > > ----------------------------------------------- > > I am sure that is because of the configurations of the IPs and the > aditional > eth0:0 and lo:0 and that's why it doesn't work, i will paste everything > that > i did, maybe, just maybe you can help me out on this one, i'm really > stuck..probably because i don't know lots of stuff on how the OSI layer is > build and how arp works > > Aditional Network conf.. > XX.XX.XX.234 (this is the lvs..) is spawned on eth1.. > so i spawned another eth1:0 > [r...@linux ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1:0 > DEVICE=eth1:0 > IPADDR=XX.XX.XX.236 # this is from the same subnet and i doesn't conflict > with an existent one > NETMASK=255.255.255.0 > NETWORK=XX.XX.XX.0 > BROADCAST=XX.XX.XX.255 > GATEWAY=XX.XX.XX.233 # i've put the same gateway as the default ip > XX.XX.XX.234 had > ONBOOT=yes > > i also enabled port forwarding > > net.ipv4.ip_forward = 1 > net.ipv4.conf.default.rp_filter = 1 > net.ipv4.conf.default.accept_source_route = 0 > kernel.sysrq = 0 > kernel.core_uses_pid = 1 > net.ipv4.tcp_syncookies = 1 > kernel.msgmnb = 65536 > kernel.msgmax = 65536 > kernel.shmmax = 68719476736 > kernel.shmall = 4294967296 > > And i did the modprobe with all those modules for IPV > > modprobe ip_vs_dh > modprobe ip_vs_ftp > modprobe ip_vs_dh > modprobe ip_vs_ftp > modprobe ip_vs > modprobe ip_vs_lblc > modprobe ip_vs_lblcr > modprobe ip_vs_lc > modprobe ip_vs_nq > modprobe ip_vs_rr > modprobe ip_vs_sed > modprobe ip_vs_sh > modprobe ip_vs_wlc > modprobe ip_vs_wrr > > and that's all i did for the LVS server.. > > Now for the webserver > XX.XX.XX.235 (this is the lvs..) is spawned on eth0.. > r...@linux ~]# cat /etc/sysconfig/network-scripts/ifcfg-lo:0 > DEVICE=lo:0 > IPADDR=XX.XX.XX.236 > NETMASK=255.255.255.255 > NETWORK=XX.XX.XX.XX.0 > BROADCAST=XX.XX.XX.255 > ONBOOT=yes > NAME=loopback > > and > > net.ipv4.ip_forward = 0 > net.ipv4.conf.lo.arp_ignore = 1 #here i have tried with eth0 instead of lo, > no luck.. > net.ipv4.conf.lo.arp_announce = 2 #here i have tried with eth0 instead of > lo, no luck.. > net.ipv4.conf.all.arp_ignore = 1 > net.ipv4.conf.all.arp_announce = 2 > net.ipv4.conf.default.rp_filter = 1 > net.ipv4.conf.default.accept_source_route = 0 > kernel.sysrq = 0 > kernel.core_uses_pid = 1 > net.ipv4.tcp_syncookies = 1 > kernel.msgmnb = 65536 > kernel.msgmax = 65536 > kernel.shmmax = 68719476736 > kernel.shmall = 4294967296 > > here i've tried with ip forward 0 and 1 , no luck the requests simply don't > reach this server only the direct ones > > Now i've understand that this is an ARP problem, and as CentOS doesn't > support the arp hidden flag on sysctl , i tried with > /etc/init.d/arptables_jf > > arptables -A IN -j DROP -d XX.XX.XX.236 > -A OUT -j mangle -o eth0 -s XX.XX.XX.234 --mangle-ip-s XX.XX.XX.236 > arptables -A OUT -j mangle -o eth0 -s XX.XX.XX.234 --mangle-ip-s > XX.XX.XX.236 > /etc/init.d/arptables_jf save > /etc/init.d/arptables_jf start > /sbin/arptables -L -v -n > > hain IN (policy ACCEPT 1353 packets, 37884 bytes) > pkts bytes target in out source-ip destination-ip source-hw destination-hw > hlen op hrd pro > 0 0 DROP * * 0.0.0.0/0 XX.XX.XX.236 00/00 00/00 any 0000/0000 0000/0000 > 0000/0000 > > Chain OUT (policy ACCEPT 25 packets, 700 bytes) > pkts bytes target in out source-ip destination-ip source-hw destination-hw > hlen op hrd pro > 0 0 mangle * eth0 XX.XX.XX.234 0.0.0.0/0 00/00 00/00 any 0000/0000 > 0000/0000 > 0000/0000 --mangle-ip-s XX.XX.XX.236 > > Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) > pkts bytes target in out source-ip destination-ip source-hw destination-hw > hlen op hrd pro > > ----------------------------------------------- > > i've probed with ipvsadm or something like it, to see the active > connections, and they are always to 0 > > i've modprobed here the same modules, no luck.. > > So from this point i'm really stuck and don't know what to do... > > Here's the ifconfig from both servers if that helps > from the LVS (xx.234) > > eth1 Link encap:Ethernet HWaddr 00:1B:21:46:3E:A9 > inet addr:XX.XX.XX.234 Bcast:XX.XX.XX.239 Mask:255.255.255.248 > inet6 addr: fe80::21b:21ff:fe45:3ea9/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:276338 errors:0 dropped:2 overruns:0 frame:0 > TX packets:220590 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:156995116 (149.7 MiB) TX bytes:30754525 (29.3 MiB) > Base address:0xd000 Memory:e2020000-e2040000 > > eth1:0 Link encap:Ethernet HWaddr 00:1B:21:46:3E:A9 > inet addr:XX.XX.XX.236 Bcast:XX.XX.XX.255 Mask:255.255.255.0 > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > Base address:0xd000 Memory:e2020000-e2040000 > > lo Link encap:Local Loopback > inet addr:127.0.0.1 Mask:255.0.0.0 > inet6 addr: ::1/128 Scope:Host > UP LOOPBACK RUNNING MTU:16436 Metric:1 > RX packets:150 errors:0 dropped:0 overruns:0 frame:0 > TX packets:150 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:16592 (16.2 KiB) TX bytes:16592 (16.2 KiB) > > here's the one from the webserver > > eth0 Link encap:Ethernet HWaddr 00:24:1D:72:61:AB > inet addr:XX.XX.XX.235 Bcast:XX.XX.XX.239 Mask:255.255.255.248 > inet6 addr: fe80::224:1dff:fe74:61ab/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:301529 errors:0 dropped:0 overruns:0 frame:0 > TX packets:255827 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:149822776 (142.8 MiB) TX bytes:36393789 (34.7 MiB) > Interrupt:233 Base address:0x2000 > > lo Link encap:Local Loopback > inet addr:127.0.0.1 Mask:255.0.0.0 > inet6 addr: ::1/128 Scope:Host > UP LOOPBACK RUNNING MTU:16436 Metric:1 > RX packets:56 errors:0 dropped:0 overruns:0 frame:0 > TX packets:56 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:4448 (4.3 KiB) TX bytes:4448 (4.3 KiB) > > lo:0 Link encap:Local Loopback > inet addr:XX.XX.XX.236 Mask:255.255.255.255 > UP LOOPBACK RUNNING MTU:16436 Metric:1 > > i've checked the firewall it's disabled , iptables stoped, selinux is > disabled.. > > I am thanking you in advance for helping solve this problem. I think it's > a > problem that many have with Fedora/Red Hat/ Cent-oS > _______________________________________________ > Please read the documentation before posting - it's available at: > http://www.linuxvirtualserver.org/ > > LinuxVirtualServer.org mailing list - [email protected] > Send requests to [email protected] > or go to http://lists.graemef.net/mailman/listinfo/lvs-users > _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - [email protected] Send requests to [email protected] or go to http://lists.graemef.net/mailman/listinfo/lvs-users
