Is there anything that you guys can suggest I do around the cache? Should I try a different store type? A different filesystem type perhaps?
If your store has a configuration knob that effectively limits disk writing rate, then use it to limit that rate to avoid overflowing the queue. You can also consider using rock store that has an explicit write rate limiting option (but comes with other problems that may or may not affect your setup). Adding more physical disk spindles helps, of course. Thanks Alex. My next approach is to add more spindles. In fact I am currently testing with 4 cache dirs. All aufs currently. After adding these cache dirs, Squid started crashing every 20mins or so.. --cut-- (Cache.log) FATAL: Received Segment Violation...dying. 2016/09/26 15:54:54 kid1| Closing HTTP port 0.0.0.0:8080 2016/09/26 15:54:54 kid1| storeDirWriteCleanLogs: Starting... 2016/09/26 15:54:54 kid1| 65536 entries written so far. 2016/09/26 15:54:54 kid1| Finished. Wrote 98159 entries. 2016/09/26 15:54:54 kid1| Took 0.05 seconds (2044425.47 entries/sec). CPU Usage: 1230.157 seconds = 166.490 user + 1063.666 sys Maximum Resident Size: 3535840 KB Page faults with physical i/o: 0 2016/09/26 15:54:57 kid1| Set Current Directory to /var/cache/squid/ 2016/09/26 15:54:57 kid1| Starting Squid Cache version 3.5.21 for x86_64-pc-linux-gnu... 2016/09/26 15:54:57 kid1| Service Name: squid 2016/09/26 15:54:57 kid1| Process ID 16054 2016/09/26 15:54:57 kid1| Process Roles: worker 2016/09/26 15:54:57 kid1| With 65536 file descriptors available 2016/09/26 15:54:57 kid1| Initializing IP Cache... 2016/09/26 15:54:57 kid1| DNS Socket created at 0.0.0.0, FD 7 2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf 2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf 2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf 2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf 2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf 2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf 2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf 2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf 2016/09/26 15:54:57 kid1| Adding nameserver xxxxxx from /etc/resolv.conf 2016/09/26 15:54:57 kid1| Adding nameserver xxxxxx from /etc/resolv.conf 2016/09/26 15:54:57 kid1| Adding nameserver xxxxxx from /etc/resolv.conf 2016/09/26 15:54:57 kid1| Logfile: opening log daemon:/var/log/squid//access.log 2016/09/26 15:54:57 kid1| Logfile Daemon: opening log /var/log/squid//access.log 2016/09/26 15:54:57 kid1| Store logging disabled 2016/09/26 15:54:57 kid1| Swap maxSize 41943040 + 4194304 KB, estimated 3549026 objects 2016/09/26 15:54:57 kid1| Target number of buckets: 177451 2016/09/26 15:54:57 kid1| Using 262144 Store buckets 2016/09/26 15:54:57 kid1| Max Mem size: 4194304 KB 2016/09/26 15:54:57 kid1| Max Swap size: 41943040 KB 2016/09/26 15:54:57 kid1| Rebuilding storage in /var/cache/squid-c1/ (clean log) 2016/09/26 15:54:57 kid1| Rebuilding storage in /var/cache/squid-c2/ (clean log) 2016/09/26 15:54:57 kid1| Rebuilding storage in /var/cache/squid-c3/ (clean log) 2016/09/26 15:54:57 kid1| Rebuilding storage in /var/cache/squid-c4/ (clean log) 2016/09/26 15:54:57 kid1| Using Least Load store dir selection 2016/09/26 15:54:57 kid1| Set Current Directory to /var/cache/squid/ 2016/09/26 15:54:57 kid1| Finished loading MIME types and icons. 2016/09/26 15:54:57 kid1| HTCP Disabled. 2016/09/26 15:54:57 kid1| Sending SNMP messages from 0.0.0.0:3401 2016/09/26 15:54:57 kid1| Configuring Parent xxxxxx8/80/0 2016/09/26 15:54:57 kid1| Configuring Parent xxxxxx/80/0 2016/09/26 15:54:57 kid1| Configuring Sibling xxxxxx/8080/3131 2016/09/26 15:54:57 kid1| Squid plugin modules loaded: 0 2016/09/26 15:54:57 kid1| Adaptation support is off. 2016/09/26 15:54:57 kid1| Accepting HTTP Socket connections at local=0.0.0.0:8080 remote=[::] FD 18 flags=9 2016/09/26 15:54:57 kid1| Accepting SNMP messages on 0.0.0.0:3401 2016/09/26 15:54:57 kid1| Store rebuilding is 33.00% complete 2016/09/26 15:54:57 kid1| Done reading /var/cache/squid-c3/ swaplog (8660 entries) 2016/09/26 15:54:57 kid1| Done reading /var/cache/squid-c1/ swaplog (12119 entries) 2016/09/26 15:54:58 kid1| Done reading /var/cache/squid-c2/ swaplog (30684 entries) 2016/09/26 15:54:58 kid1| Done reading /var/cache/squid-c4/ swaplog (46696 entries) 2016/09/26 15:54:58 kid1| Finished rebuilding storage from disk. 2016/09/26 15:54:58 kid1| 98159 Entries scanned 2016/09/26 15:54:58 kid1| 0 Invalid entries. 2016/09/26 15:54:58 kid1| 0 With invalid flags. 2016/09/26 15:54:58 kid1| 98159 Objects loaded. 2016/09/26 15:54:58 kid1| 0 Objects expired. 2016/09/26 15:54:58 kid1| 0 Objects cancelled. 2016/09/26 15:54:58 kid1| 0 Duplicate URLs purged. 2016/09/26 15:54:58 kid1| 0 Swapfile clashes avoided. 2016/09/26 15:54:58 kid1| Took 0.61 seconds (162197.49 objects/sec). 2016/09/26 15:54:58 kid1| Beginning Validation Procedure 2016/09/26 15:54:58 kid1| Completed Validation Procedure 2016/09/26 15:54:58 kid1| Validated 98159 Entries 2016/09/26 15:54:58 kid1| store_swap_size = 11495920.00 KB 2016/09/26 15:54:58 kid1| storeLateRelease: released 0 objects --cut-- I tried rock but was stuck with this error for a while and decided to go back to aufs until I can do some more reading up. FATAL: Rock cache_dir at /var/cache/squid-c3//rock failed to open db file: (2) No such file or directory I am currently testing again after recompiling squid.. The rest of my proxy cluster has the "cache deny all" configuration directive.. So nothing is cached. Disclaimer: http://www.shopriteholdings.co.za/Pages/ShopriteE-mailDisclaimer.aspx
_______________________________________________ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
