On 16 February 2011 02:56, Roger Littin <ro...@rlmedia.co.nz> wrote: > Hi, > > I have a set up using ldirectord in direct routing to 30 media servers. > > The director is a dell R210 4 core with 8GB memory and has 2 1Gbps network > connections, 1 public & 1 private. > > Each real server is a Dell R610 16 core with 12GB memory and has the same > network connections as above. > > All the servers are running centos 5. > > According to the data center, these servers are connected to pairs of 40Gb > switches and there is ample capacity. > > What happens is when the connections per real server get to around 1000 – > 1200 concurrent connections, the bandwidth outgoing per server wont go above > about 250Mbps which relates to about 7.5Gbps across all servers. At that > time is when the complaints start coming in about stream problems. > > I guess the question is could the director somehow be limiting the throughput > on the real servers or is the dc not telling the truth about? > The bandwidth going through the nics on the director is around 50 – 75Mbps in > on the public nic and out on the private nic to the real servers. > > Before we started using lvs, we had 10 servers running with round robin dns > and these would easily handle 900Mbps each at the same time. > > lidrectord config file. > > # Global Directives > checktimeout=10 > checkinterval=5 > #fallback=127.0.0.1:80 > autoreload=yes > callback="/etc/ha.d/syncsettings.sh" > logfile="/var/log/ldirectord.log" > #logfile="local0" > #emailalert="admin@x.y.z" > #emailalertfreq=3600 > #emailalertstatus=all > quiescent=no > > virtual=147 > real=172.31.214.12 gate 100 > real=172.31.214.13 gate 100 > real=172.31.214.14 gate 100 > real=172.31.214.15 gate 100 > real=172.31.214.16 gate 100 > real=172.31.214.17 gate 100 > real=172.31.214.18 gate 100 > real=172.31.214.19 gate 100 > real=172.31.214.21 gate 100 > real=172.31.214.22 gate 100 > real=172.31.214.23 gate 100 > real=172.31.214.24 gate 100 > real=172.31.214.25 gate 100 > real=172.31.214.26 gate 100 > real=172.31.214.28 gate 100 > real=172.31.214.29 gate 100 > real=172.31.214.30 gate 100 > real=172.31.214.31 gate 100 > real=172.31.214.32 gate 100 > real=172.31.214.33 gate 100 > real=172.31.214.34 gate 100 > real=172.31.214.35 gate 100 > real=172.31.214.36 gate 100 > real=172.31.214.37 gate 100 > real=172.31.214.38 gate 100 > real=172.31.214.39 gate 100 > real=172.31.214.40 gate 100 > real=172.31.214.41 gate 100 > real=172.31.214.42 gate 100 > scheduler=wlc > protocol=fwm > persistent=60 > netmask=255.255.255.255 > service=http > checkport=1935 > request="/" > receive="Wowza Media Server 2" > > iptables > > *mangle > :PREROUTING ACCEPT [438:421747] > :INPUT ACCEPT [438:421747] > :FORWARD ACCEPT [0:0] > :OUTPUT ACCEPT [95:14749] > :POSTROUTING ACCEPT [122:21354] > -A PREROUTING –d *.*.*.147 -p tcp -m tcp --dport 80 -j MARK --set-mark 0x93 > -A PREROUTING -d *.*.*.147 -p tcp -m tcp --dport 443 -j MARK --set-mark 0x93 > -A PREROUTING -d *.*.*.147 -p tcp -m tcp --dport 554 -j MARK --set-mark 0x93 > -A PREROUTING -d *.*.*.147 -p tcp -m tcp --dport 1935 -j MARK --set-mark 0x93 > -A PREROUTING -d *.*.*.147 -p udp -m udp --dport 6970:9999 -j MARK --set-mark > 0x93 > COMMIT > > the main port that is used is 1935. > > Thanks in advance. > > Regards, > > Roger. > _______________________________________________ > Please read the documentation before posting - it's available at: > http://www.linuxvirtualserver.org/ > > LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org > Send requests to lvs-users-requ...@linuxvirtualserver.org > or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Roger, My gut would say that your switch is getting saturated (not the director). I wonder if you could get some of the servers to reply to a different switch (local subnet) but still through the director i.e. prove the director can can more load if the outgoing traffic is through a different switch. Also CPU load on the director is a pretty good indicator of stress. We've had customers doing similar kind of load (but every setup is different). -- Regards, Malcolm Turnbull. Loadbalancer.org Ltd. Phone: +44 (0)870 443 8779 http://www.loadbalancer.org/ _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org Send requests to lvs-users-requ...@linuxvirtualserver.org or go to http://lists.graemef.net/mailman/listinfo/lvs-users
