Dear all,
I have a Squid transparent proxy which uses 40 Squirm URL rewriter
processes. Everything worked fine until today, when Squid crashed
after an error "179 pending requests queued":
2011/11/15 13:55:41| WARNING: All redirector processes are busy.
2011/11/15 13:55:41| WARNING: 40 pending requests queued
2011/11/15 13:55:41| Consider increasing the number of redirector
processes in your config file.
2011/11/15 14:05:41| WARNING: All redirector processes are busy.
2011/11/15 14:05:41| WARNING: 179 pending requests queued
2011/11/15 14:05:41| storeDirWriteCleanLogs: Starting...
2011/11/15 14:05:41| WARNING: Closing open FD 190
2011/11/15 14:05:41| 65536 entries written so far.
2011/11/15 14:05:41| 131072 entries written so far.
2011/11/15 14:05:41| 196608 entries written so far.
2011/11/15 14:05:41| 262144 entries written so far.
2011/11/15 14:05:41| 327680 entries written so far.
2011/11/15 14:05:41| 393216 entries written so far.
2011/11/15 14:05:41| 458752 entries written so far.
2011/11/15 14:05:41| 524288 entries written so far.
2011/11/15 14:05:41| 589824 entries written so far.
2011/11/15 14:05:42| Finished. Wrote 633017 entries.
2011/11/15 14:05:42| Took 1.49 seconds (426234.18 entries/sec).
FATAL: Too many queued redirector requests
Squid Cache (Version 3.1.7): Terminated abnormally.
CPU Usage: 91414.241 seconds = 55144.038 user + 36270.203 sys
Maximum Resident Size: 0 KB
Page faults with physical i/o: 8
Memory usage for squid via mallinfo():
total space in arena: 1007116 KB
Ordinary blocks: 1003347 KB 100368 blks
Small blocks: 0 KB 1 blks
Holding blocks: 27164 KB 8 blks
Free Small blocks: 0 KB
Free Ordinary blocks: 3768 KB
Total in use: 1030511 KB 102%
Total free: 3768 KB 0%
Restarting gave this error:
2011/11/15 14:14:02| WARNING: Disk space over limit: 11037556 KB > 10240000 KB
2011/11/15 14:14:02| IpIntercept.cc(137) NetfilterInterception: NF
getsockopt(SO_ORIGINAL_DST) failed on FD 10: (2) No such file or
directory
2011/11/15 14:14:02| IpIntercept.cc(137) NetfilterInterception: NF
getsockopt(SO_ORIGINAL_DST) failed on FD 12: (2) No such file or
directory
2011/11/15 14:14:02| IpIntercept.cc(137) NetfilterInterception: NF
getsockopt(SO_ORIGINAL_DST) failed on FD 14: (2) No such file or
directory
2011/11/15 14:14:02| IpIntercept.cc(137) NetfilterInterception: NF
getsockopt(SO_ORIGINAL_DST) failed on FD 16: (2) No such file or
directory
2011/11/15 14:14:02| IpIntercept.cc(137) NetfilterInterception: NF
getsockopt(SO_ORIGINAL_DST) failed on FD 18: (2) No such file or
directory
(... snipped 60 entries similar as the one above ... )
2011/11/15 14:14:05| WARNING: All redirector processes are busy.
2011/11/15 14:14:05| WARNING: 40 pending requests queued
2011/11/15 14:14:05| Consider increasing the number of redirector
processes in your config file.
I tried increasing redirect_children to 300 in squid.conf, but since
each children spawns as a process, this brought the system to its
knees. Even with no active HTTP clients.
What is the max/reasonable number of Squirm redirect_children?
These two lines also puzzle me. I could rebuild the cache to get rid
of it, but I'd like to know why this occurred:
2011/11/15 14:14:02| WARNING: Disk space over limit: 11037556 KB > 10240000 KB
Total in use: 1030511 KB 102%
Thanks a lot for your thoughts,
Leonardo
