Keepalives did the trick. Cheers! From: Nathan Williams [mailto:[email protected]] Sent: Tuesday, March 24, 2015 16:02 To: Jim Gronowski Cc: [email protected] Subject: Re: timeout values for redis?
i should probably add... setting all members to backups means of course that only the first server to pass the check will actually receive traffic (unless you're using "option allbackups"). this works perfectly for us, but may not work for you depending on your use-case. On Tue, Mar 24, 2015 at 3:55 PM, Nathan Williams <[email protected]<mailto:[email protected]>> wrote: Hey Jim, Here's the configuration we're using for our redis pool: defaults TCP mode tcp log global option tcplog option clitcpka option srvtcpka timeout connect 5s timeout client 300s timeout server 300s source 12.34.56.78 listen redis bind 0.0.0.0:6379<http://0.0.0.0:6379> option tcp-check tcp-check send PING\r\n tcp-check expect string +PONG tcp-check send info\ replication\r\n tcp-check expect string role:master tcp-check send QUIT\r\n tcp-check expect string +OK server redis01.prod 12.34.56.79:6379<http://12.34.56.79:6379> backup check inter 1000 rise 2 fall 5 server redis02.prod 12.34.56.80:6379<http://12.34.56.80:6379> backup check inter 1000 rise 2 fall 5 The key items for silencing client errors were the tcpka (keepalive) configurations, along with setting the servers all to backups, which helped us avoid clients briefly getting connected to the read-only slave immediately following an haproxy reload/restart. hope that helps! regards, Nathan W On Tue, Mar 24, 2015 at 3:48 PM, Ha Quan Le <[email protected]<mailto:[email protected]>> wrote: Thanks, I sent request previously to you but I have done it. Ha. ________________________________ From: "Jim Gronowski" <[email protected]<mailto:[email protected]>> To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Sent: Tuesday, March 24, 2015 1:25:33 PM Subject: timeout values for redis? Does anyone have any feedback on sane timeout values for load balancing redis? The testing config I was using had ‘timeout client 50000’ and I was getting consistent client disconnects in the logs. I increased it to two minutes and things have improved significantly, though I do see client disconnects every few hours (but the application is behaving normally). Client is StackExchange.Redis if that helps. Google wasn’t much use. HA-Proxy version 1.5.10. Full config: global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy stats socket /run/haproxy/admin.sock mode 660 level admin stats timeout 30s user haproxy group haproxy daemon defaults log global mode tcp option tcplog option dontlognull timeout connect 5000 timeout client 2m timeout server 120000 errorfile 400 /etc/haproxy/errors/400.http errorfile 403 /etc/haproxy/errors/403.http errorfile 408 /etc/haproxy/errors/408.http errorfile 500 /etc/haproxy/errors/500.http errorfile 502 /etc/haproxy/errors/502.http errorfile 503 /etc/haproxy/errors/503.http errorfile 504 /etc/haproxy/errors/504.http frontend redisFE bind *:6379 mode tcp maxconn 10240 default_backend redisBE backend redisBE mode tcp option tcplog balance source option tcp-check tcp-check send PING\r\n tcp-check expect string +PONG tcp-check send info\ replication\r\n tcp-check expect string role:master tcp-check send QUIT\r\n tcp-check expect string +OK server A-redis-01 X:6379 maxconn 1024 check inter 1s server A-redis-02 X:6379 maxconn 1024 check inter 1s server B-redis-01 X:6379 maxconn 1024 check inter 1s server B-redis-02 X:6379 maxconn 1024 check inter 1s Jim Gronowski Network Administrator DiTronics, LLC. Ditronics, LLC email disclaimer: This communication, including attachments, is intended only for the exclusive use of addressee and may contain proprietary, confidential, or privileged information. Any use, review, duplication, disclosure, dissemination, or distribution is strictly prohibited. If you were not the intended recipient, you have received this communication in error. Please notify sender immediately by return e-mail, delete this communication, and destroy any copies.
