Wha't i'm trying to do is setup a load banancer using 3 servers for a webservice (HTTP)
1st - the main server which will redirect to the 2nd or 3rd server. I called this LVS (and it's where ldirectord is) 2nd - runs a simple http service and when a client enters...and puts the ip, or domain of the 1st server, he should get the contents from the 2nd server (if up). eighterwise the content from the 3rd server 3rd - runs the same http service.. the only thing different from the ALL tutorials that i've read is that those 3 servers are on the internet , with public ips not behind some private 192.168.xx network 2 of the ip's are in the same subbet of a class C 1 is not even in the same continent :) So a relatively simple thing to do , that ldirectord doesn't do..in this case ---------------------- I have 3 PUBLIC ip's like (none are with 192..) , two of them are on the same subnet, I'm running Cent-oS 5, and here is a COMPLETE list of what i have done. XX.XX.XX.234 (this is the lvs..) XX.XX.XX.235 real server (web ngix) YYY.YYY.YYY.163 real server (web apache) - I don't really care about this one, i can move it into the we will use XX.XX.XX.236 as virtual..(there isn't an ip on the net with that number up). of course it will be simpler to use a 192.. but, i have tried that also ,and no luck same subnet as the first 2 ones, i just want to make it work from the 234 -> 235..but it gives me a timeout on the browser... Here's what i did: [r...@linux ~]# cat /etc/ha.d/ldirectord.cf checktimeout=3 checkinterval=10 autoreload=yes logfile="/var/log/ldirectord.log" quiescent=no virtual=XX.XX.XX.236:80 fallback=127.0.0.1:80 real=XX.XX.XX.235:80 gate real=YYY.YYY.YYY.235:80 gate service=http request="test.html" receive="Still alive" scheduler=rr #here i've tried with wlr as well protocol=tcp checktype=negotiate [r...@linux ~]# /usr/sbin/ldirectord -d /etc/ha.d/ldirectord.cf start DEBUG2: Running exec(/usr/sbin/ldirectord -d /etc/ha.d/ldirectord.cf start) Running exec(/usr/sbin/ldirectord -d /etc/ha.d/ldirectord.cf start) DEBUG2: Invoking ldirectord invoked as: /usr/sbin/ldirectord /etc/ha.d/ ldirectord.cf start Invoking ldirectord invoked as: /usr/sbin/ldirectord /etc/ha.d/ldirectord.cfstart DEBUG2: Starting Linux Director v1.186-ha-2.1.4 with pid: 18619 Starting Linux Director v1.186-ha-2.1.4 with pid: 18619 DEBUG2: Running system(/sbin/ipvsadm -A -t XX.XX.XX.236:80 -s rr ) Running system(/sbin/ipvsadm -A -t XX.XX.XX.236:80 -s rr ) DEBUG2: Added virtual server: XX.XX.XX.236:80 Added virtual server: XX.XX.XX.236:80 DEBUG2: Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r 127.0.0.1:80-g -w 1) Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r 127.0.0.1:80 -g -w 1) DEBUG2: Added fallback server: 127.0.0.1:80 (XX.XX.XX.236:80) (Weight set to 1) Added fallback server: 127.0.0.1:80 (XX.XX.XX.236:80) (Weight set to 1) DEBUG2: Disabled real server=negotiate:http:tcp:XX.XX.XX.235:80:::1:gate:\/test\.html:Still\ alive (virtual=tcp:XX.XX.XX.236:80) DEBUG2: Checking negotiate: real server=negotiate:http:tcp:XX.XX.XX.235:80:::1:gate:\/test\.html:Still\ alive (virtual=tcp:XX.XX.XX.236:80) DEBUG2: check_http: url="http://XX.XX.XX.235:80/test.html"; virtualhost="XX.XX.XX.235" LWP::UserAgent::new: () DEBUG2: Starting Check DEBUG2: Starting HTTP/HTTPS LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://XX.XX.XX.235:80/test.html<http://xx.xx.xx.235/test.html> LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::collect: read 12 bytes LWP::UserAgent::request: Simple response: OK DEBUG2: Finished HTTP/HTTPS DEBUG2: Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w 1) Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w 1) DEBUG2: system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w 1) failed: No child processes system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w 1) failed: No child processes DEBUG2: Added real server: XX.XX.XX.235:80 (XX.XX.XX.236:80) (Weight set to 1) Added real server: XX.XX.XX.235:80 (XX.XX.XX.236:80) (Weight set to 1) DEBUG2: Running system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80) Running system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80) DEBUG2: system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80) failed: No child processes system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80) failed: No child processes DEBUG2: Deleted fallback server: 127.0.0.1:80 (XX.XX.XX.236:80) Deleted fallback server: 127.0.0.1:80 (XX.XX.XX.236:80) DEBUG2: Enabled real server=negotiate:http:tcp:XX.XX.XX.235:80:::1:gate:\/test\.html:Still\ alive (virtual=tcp:XX.XX.XX.236:80) DEBUG2: check_http: http://XX.XX.XX.235:80/test.html<http://xx.xx.xx.235/test.html>is up If i go to the webserver i can see that ldirector is actually testing the test.html..every 10 seconds like in the conf ----------------------------------------------- I am sure that is because of the configurations of the IPs and the aditional eth0:0 and lo:0 and that's why it doesn't work, i will paste everything that i did, maybe, just maybe you can help me out on this one, i'm really stuck..probably because i don't know lots of stuff on how the OSI layer is build and how arp works Aditional Network conf.. XX.XX.XX.234 (this is the lvs..) is spawned on eth1.. so i spawned another eth1:0 [r...@linux ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1:0 DEVICE=eth1:0 IPADDR=XX.XX.XX.236 # this is from the same subnet and i doesn't conflict with an existent one NETMASK=255.255.255.0 NETWORK=XX.XX.XX.0 BROADCAST=XX.XX.XX.255 GATEWAY=XX.XX.XX.233 # i've put the same gateway as the default ip XX.XX.XX.234 had ONBOOT=yes i also enabled port forwarding net.ipv4.ip_forward = 1 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 0 kernel.core_uses_pid = 1 net.ipv4.tcp_syncookies = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.shmmax = 68719476736 kernel.shmall = 4294967296 And i did the modprobe with all those modules for IPV modprobe ip_vs_dh modprobe ip_vs_ftp modprobe ip_vs_dh modprobe ip_vs_ftp modprobe ip_vs modprobe ip_vs_lblc modprobe ip_vs_lblcr modprobe ip_vs_lc modprobe ip_vs_nq modprobe ip_vs_rr modprobe ip_vs_sed modprobe ip_vs_sh modprobe ip_vs_wlc modprobe ip_vs_wrr and that's all i did for the LVS server.. Now for the webserver XX.XX.XX.235 (this is the lvs..) is spawned on eth0.. r...@linux ~]# cat /etc/sysconfig/network-scripts/ifcfg-lo:0 DEVICE=lo:0 IPADDR=XX.XX.XX.236 NETMASK=255.255.255.255 NETWORK=XX.XX.XX.XX.0 BROADCAST=XX.XX.XX.255 ONBOOT=yes NAME=loopback and net.ipv4.ip_forward = 0 net.ipv4.conf.lo.arp_ignore = 1 #here i have tried with eth0 instead of lo, no luck.. net.ipv4.conf.lo.arp_announce = 2 #here i have tried with eth0 instead of lo, no luck.. net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 0 kernel.core_uses_pid = 1 net.ipv4.tcp_syncookies = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.shmmax = 68719476736 kernel.shmall = 4294967296 here i've tried with ip forward 0 and 1 , no luck the requests simply don't reach this server only the direct ones Now i've understand that this is an ARP problem, and as CentOS doesn't support the arp hidden flag on sysctl , i tried with /etc/init.d/arptables_jf arptables -A IN -j DROP -d XX.XX.XX.236 -A OUT -j mangle -o eth0 -s XX.XX.XX.234 --mangle-ip-s XX.XX.XX.236 arptables -A OUT -j mangle -o eth0 -s XX.XX.XX.234 --mangle-ip-s XX.XX.XX.236 /etc/init.d/arptables_jf save /etc/init.d/arptables_jf start /sbin/arptables -L -v -n hain IN (policy ACCEPT 1353 packets, 37884 bytes) pkts bytes target in out source-ip destination-ip source-hw destination-hw hlen op hrd pro 0 0 DROP * * 0.0.0.0/0 XX.XX.XX.236 00/00 00/00 any 0000/0000 0000/0000 0000/0000 Chain OUT (policy ACCEPT 25 packets, 700 bytes) pkts bytes target in out source-ip destination-ip source-hw destination-hw hlen op hrd pro 0 0 mangle * eth0 XX.XX.XX.234 0.0.0.0/0 00/00 00/00 any 0000/0000 0000/0000 0000/0000 --mangle-ip-s XX.XX.XX.236 Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target in out source-ip destination-ip source-hw destination-hw hlen op hrd pro ----------------------------------------------- i've probed with ipvsadm or something like it, to see the active connections, and they are always to 0 i've modprobed here the same modules, no luck.. So from this point i'm really stuck and don't know what to do... Here's the ifconfig from both servers if that helps from the LVS (xx.234) eth1 Link encap:Ethernet HWaddr 00:1B:21:46:3E:A9 inet addr:XX.XX.XX.234 Bcast:XX.XX.XX.239 Mask:255.255.255.248 inet6 addr: fe80::21b:21ff:fe45:3ea9/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:276338 errors:0 dropped:2 overruns:0 frame:0 TX packets:220590 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:156995116 (149.7 MiB) TX bytes:30754525 (29.3 MiB) Base address:0xd000 Memory:e2020000-e2040000 eth1:0 Link encap:Ethernet HWaddr 00:1B:21:46:3E:A9 inet addr:XX.XX.XX.236 Bcast:XX.XX.XX.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Base address:0xd000 Memory:e2020000-e2040000 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:150 errors:0 dropped:0 overruns:0 frame:0 TX packets:150 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:16592 (16.2 KiB) TX bytes:16592 (16.2 KiB) here's the one from the webserver eth0 Link encap:Ethernet HWaddr 00:24:1D:72:61:AB inet addr:XX.XX.XX.235 Bcast:XX.XX.XX.239 Mask:255.255.255.248 inet6 addr: fe80::224:1dff:fe74:61ab/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:301529 errors:0 dropped:0 overruns:0 frame:0 TX packets:255827 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:149822776 (142.8 MiB) TX bytes:36393789 (34.7 MiB) Interrupt:233 Base address:0x2000 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:56 errors:0 dropped:0 overruns:0 frame:0 TX packets:56 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:4448 (4.3 KiB) TX bytes:4448 (4.3 KiB) lo:0 Link encap:Local Loopback inet addr:XX.XX.XX.236 Mask:255.255.255.255 UP LOOPBACK RUNNING MTU:16436 Metric:1 i've checked the firewall it's disabled , iptables stoped, selinux is disabled.. I am thanking you in advance for helping solve this problem. I think it's a problem that many have with Fedora/Red Hat/ Cent-oS _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - [email protected] Send requests to [email protected] or go to http://lists.graemef.net/mailman/listinfo/lvs-users
