Hey guys - we're using mod_proxy on 2 Dell Poweredge Load Balancers. We're
receiving some odd errors at random. Googling or reading the manual (RTFM)
is not providing any fruitful fixes.
We switched between LB1 and LB2 but ran into the same error on each. The
more traffic we get, the higher the chance of this error.
*Do any of you know a good solution to mod_proxy throwing the following
error?*
[Wed Sep 23 21:42:25 2009] [error] [client xx.xx.xx.xx] proxy: DNS lookup
failure for: web1. domain.com returned by /file.php
[Wed Sep 23 21:42:31 2009] [error] [client xx.xx.xx.xx ] proxy: DNS lookup
failure for: web1. domain.com returned by /file.php
[Wed Sep 23 21:42:33 2009] [error] [client xx.xx.xx.xx] proxy: DNS lookup
failure for: web1. domain.com returned by /file.php
[Wed Sep 23 21:42:34 2009] [error] [client xx.xx.xx.xx] proxy: DNS lookup
failure for: web2. domain.com returned by /file.php
[Wed Sep 23 21:42:36 2009] [error] [client xx.xx.xx.xx] proxy: DNS lookup
failure for: web1. domain.com returned by /file.php
[Wed Sep 23 21:42:41 2009] [error] [client xx.xx.xx.xx] proxy: DNS lookup
failure for: web2. domain.com returned by /file.php
[Wed Sep 23 21:42:47 2009] [error] (70007)The timeout specified has expired:
proxy: HTTP: attempt to connect to 10.3.1.30:80 (web1. domain.com) failed
[Wed Sep 23 21:42:47 2009] [error] ap_proxy_connect_backend disabling worker
for (web1. domain.com)
[Wed Sep 23 21:42:47 2009] [error] (70007)The timeout specified has expired:
proxy: HTTP: attempt to connect to 10.3.1.31:80 (web2. domain.com) failed
[Wed Sep 23 21:42:47 2009] [error] ap_proxy_connect_backend disabling worker
for (web2. domain.com)
[Wed Sep 23 21:42:47 2009] [error] (70007)The timeout specified has expired:
proxy: HTTP: attempt to connect to 10.3.1.31:80 (web2. domain.com) failed
[Wed Sep 23 21:42:47 2009] [error] ap_proxy_connect_backend disabling worker
for (web2. domain.com)
[Wed Sep 23 21:42:47 2009] [error] (70007)The timeout specified has expired:
proxy: HTTP: attempt to connect to 10.3.1.31:80 (web2. domain.com) failed
[Wed Sep 23 21:42:47 2009] [error] (70007)The timeout specified has expired:
proxy: HTTP: attempt to connect to 10.3.1.30:80 (web1. domain.com) failed
[Wed Sep 23 21:42:47 2009] [error] ap_proxy_connect_backend disabling worker
for (web1. domain.com)
*Here's output of our netstat -s*
netstat -s
Ip:
24232547 total packets received
115 forwarded
0 incoming packets discarded
21415023 incoming packets delivered
14100744 requests sent out
1854 outgoing packets dropped
Icmp:
61194 ICMP messages received
0 input ICMP message failed.
ICMP input histogram:
destination unreachable: 62
echo requests: 61131
echo replies: 1
62204 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
destination unreachable: 1072
echo request: 1
echo replies: 61131
IcmpMsg:
InType0: 1
InType3: 62
InType8: 61131
OutType0: 61131
OutType3: 1072
OutType8: 1
Tcp:
63102 active connections openings
600330 passive connection openings
125 failed connection attempts
260826 connection resets received
5 connections established
12624470 segments received
10931112 segments send out
3841 segments retransmited
0 bad segments received.
6724 resets sent
Udp:
5918995 packets received
135 packets to unknown port received.
0 packet receive errors
3106510 packets sent
TcpExt:
493 invalid SYN cookies received
125 resets received for embryonic SYN_RECV sockets
16 packets pruned from receive queue because of socket buffer overrun
56479 TCP sockets finished time wait in fast timer
5 time wait sockets recycled by time stamp
431143 delayed acks sent
76 delayed acks further delayed because of locked socket
Quick ack mode was activated 551 times
5856 times the listen queue of a socket overflowed
5856 SYNs to LISTEN sockets ignored
1200246 packets directly queued to recvmsg prequeue.
82 packets directly received from prequeue
5327501 packets header predicted
1759302 acknowledgments not containing data received
3582614 predicted acknowledgments
295 times recovered from packet loss due to SACK data
Detected reordering 5 times using FACK
Detected reordering 1 times using SACK
Detected reordering 5 times using time stamp
3 congestion windows fully recovered
11 congestion windows partially recovered using Hoe heuristic
TCPDSACKUndo: 48
1350 congestion windows recovered after partial ack
426 TCP data loss events
64 timeouts after SACK recovery
18 timeouts in loss state
710 fast retransmits
83 forward retransmits
669 retransmits in slow start
2209 other TCP timeouts
44 sack retransmits failed
6171 packets collapsed in receive queue due to low socket buffer
558 DSACKs sent for old packets
1 DSACKs sent for out of order packets
946 DSACKs received
229 connections reset due to unexpected data
260439 connections reset due to early user close
6 connections aborted due to timeout
IpExt:
InMcastPkts: 1198
OutMcastPkts: 450
InBcastPkts: 8437124
OutBcastPkts: 2812699
--
Take care,
William Attwood
Idea Extraordinaire
[email protected]
Joan Crawford<http://www.brainyquote.com/quotes/authors/j/joan_crawford.html>
- "I, Joan Crawford, I believe in the dollar. Everything I earn, I
spend."
/*
PLUG: http://plug.org, #utah on irc.freenode.net
Unsubscribe: http://plug.org/mailman/options/plug
Don't fear the penguin.
*/
