Thank again Amos for replying.
When the cache was being rebuilt (creating the swap.state file) I didn't
have any connection through squid and still was taking too much time:
Over 24hrs.
I had to delete the entire cache, something I really didn't want to do.
Also, upgraded to squid 3.1.14. Hope that version works better.
Thanks.
On 7/8/2011 1:04 AM, Amos Jeffries wrote:
On 08/07/11 02:58, Wilson Hernandez wrote:
On 7/7/2011 10:42 AM, Wilson Hernandez wrote:
<snip>
Forgot to mention:
I am running squid along with squidguard and I don't think it messes
with squid's cache directory... This cache was used by an older squid
version (3.1.9) that maybe messed it up.
Any recommendation?
SG does not play with Squids cache.
It could be 3.1.9 ignoring absent Date: headers and letting those into
the cache. In which case you should see those particular warnings
reduce as the bad objects are found.
On 08/07/11 02:36, Wilson Hernandez wrote:
>
> I did not override the waiting with -F. This cache directory is 200GB
> not in the TB. I have this system running on a 3.2Ghz, 4GB RAM machine
I refer you back to Squids message:
"WARNING: Disk space over limit: 3615298144 KB > 204800000 KB"
in other words:
squid.conf says 200000 MB (~196GB). But Squid found 3.6 TB of stored
objects in the cache_dir.
I suspect this is a symptom of traffic going faster than Squids erase
mechanisms can cope with. So lots of supposedly erased files are left
on disk and "recovered" during the DIRTY scan.
Only faster disks can fix this for the current Squid.
> and is taking for ever to rebuild the cache (almost 20hrs), don't know
> why is taking so looong. I am thinking of deleting the entire cache
but,
> don't want to loose it all.
Squid is doing a "DIRTY" scan. It opens every folder, every file
inside, loads teh fil data parses it into the index and oves on to the
next one.
There are 3.2 millions objects to do this for. At the same time as
handling live traffic which is adding new ones rapidly. 10-20 hours is
fairly normal for a 200 GB cache of small objects.
The best way to avoid this lag in older Squid is to let it complete,
save the results into swap.state. And to ensure that shutdown_timeout
is long enough that swap.state is fully saved on shutdown in future.
You can use a workaround of starting an instance of Squid without
caching and letting clients use that directly. Starting a second in
the background to do the caching. With a bit of config tweaking the
first one can bypass the second until the cache is ready and available.
The squid-3.2 "rock" store has some fixes trying to improve the
situation. But I'm not yet sure how successful they are yet.
On 08/07/11 02:58, Wilson Hernandez wrote:
Just installed ver 3.1.14 and get the same results, no internet and is
taking a looong time for swap.state to complete.
2011/07/07 10:52:27| Starting Squid Cache version 3.1.14 for
i686-pc-linux-gnu...
2011/07/07 10:52:27| Process ID 13441
2011/07/07 10:52:27| With 32768 file descriptors available
2011/07/07 10:52:27| Initializing IP Cache...
2011/07/07 10:52:27| DNS Socket created at [::], FD 7
2011/07/07 10:52:27| DNS Socket created at 0.0.0.0, FD 8
2011/07/07 10:52:27| Adding nameserver 172.16.0.2 from squid.conf
2011/07/07 10:52:27| Adding nameserver 172.16.0.1 from squid.conf
2011/07/07 10:52:27| helperOpenServers: Starting 10/10 'zapchain'
processes
2011/07/07 10:52:27| Unlinkd pipe opened on FD 32
2011/07/07 10:52:27| Store logging disabled
2011/07/07 10:52:27| Swap maxSize 204800000 + 1048576 KB, estimated
3216384 objects
2011/07/07 10:52:27| Target number of buckets: 160819
2011/07/07 10:52:27| Using 262144 Store buckets
2011/07/07 10:52:27| Max Mem size: 1048576 KB
2011/07/07 10:52:27| Max Swap size: 204800000 KB
2011/07/07 10:52:27| Rebuilding storage in /var2/squid/cache (DIRTY)
2011/07/07 10:52:27| Using Round Robin store dir selection
2011/07/07 10:52:27| Current Directory is /workingdir/squid-3.1.14
2011/07/07 10:52:27| Loaded Icons.
2011/07/07 10:52:27| Accepting intercepted HTTP connections at
172.16.0.1:3128, FD 34.
2011/07/07 10:52:27| HTCP Disabled.
2011/07/07 10:52:27| Squid plugin modules loaded: 0
2011/07/07 10:52:27| Adaptation support is off.
2011/07/07 10:52:27| Ready to serve requests.
Amos
