Re: [squid-users] How often should I restart Squid?
On Mon, 21 Jun 2004, Lizzy Dizzy wrote: Is there any guideline as to how often do we need to restart squid (e.g. shutdown and start squid)? Normally you do not need to restart Squid, but it is probably a good idea to have Squid restarted once per month or so... The reason is that I found out that squid uses up my 2.5GB of RAM leaving only 6MB. Also it uses up half of my 500MB swapspace. Have you read the Squid FAQ on memory usage? Especially the last entry with guidelines on required amount of RAM? Regards Henrik
Re: [squid-users] How often should I restart Squid?
Matus UHLAR - fantomas [EMAIL PROTECTED] writes: On 22.06 22:47, Hendrik Voigtlnder wrote: With growing I mean the squid process increasing over the time. If I read the FAQ correctly this could be caused e.g. if more and more objects go in the the cache_dirs thus increasing the space needed for the index, i.e. if squid starts with a clean cache_dir. However, my squid is no longer increasing in process size, cache dirs are full the load is the same all day. Snapshot from top (idle squid at night). 21603 proxy 9 0 1017M 1.0G 1180 S 0.0 50.3 1:46 squid It uses roughly 50% of the RAM (machine has 2GB) , the rest is used by other processes and buffers/cache. I just think it is dangerous to disable swap, if one doesn't know how large the squid process will get, i.e. probably larger than the physical memory and this causing the OS to kill processes randomly (I had this problem with java-stuff eating up all memory). I do not think so. The same can happen if you have swap or not - it's always used as virtual memory. Having more VM just delays running machina out of it, which only happens if there's some memory leak in squid, libraries or other applications running on that machine. A small memory leak is almost unavoidable due to the phenomenon if heap fragmentation. Squid si very good at trying to avoid it (mempools) and is getting even better in version 3.0. -- kinkie (kinkie-squid [at] kinkie [dot] it) Random fortune, unrelated to the message: A Smith Wesson beats four aces.
Re: [squid-users] How often should I restart Squid?
On 25.06 22:19, Bruno Marcondes wrote: I had this weird behavior with squid on a RH9 server using kernel 2.4.22 , 2Gb of memory (the server was DL380 G3 from Compaq/HP), the swap usage was slowly growing forever and I even try disabling it with no harms , but I didn't want do it ...why ? It shouldn't swap ! it did not swap. It is just linux kernel behaviour that it saves some memory pages onto swap even if there's no need for it. IT is an optimalization feature so that makes nothing bad for you. If you have enough memory, you really can turn off swap. -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. LSD will make your ECS screen display 16.7 million colors
Re: [squid-users] How often should I restart Squid?
On Mon, 28 Jun 2004 14:05:01 +0200, Matus UHLAR - fantomas [EMAIL PROTECTED] wrote: On 25.06 22:19, Bruno Marcondes wrote: I had this weird behavior with squid on a RH9 server using kernel 2.4.22 , 2Gb of memory (the server was DL380 G3 from Compaq/HP), the swap usage was slowly growing forever and I even try disabling it with no harms , but I didn't want do it ...why ? It shouldn't swap ! it did not swap. It is just linux kernel behaviour that it saves some memory pages onto swap even if there's no need for it. IT is an optimalization feature so that makes nothing bad for you. If you have enough memory, you really can turn off swap. But it was consuming swap resources without ever freeing it. I take care of a lot of other linux boxes running apache+mod_mem_cache , apache + jrun/jetty + java and other memory consuming applications and have never seen this swap indefinitely growing usage behavior on then. After all kernel-2.6.6 gave my squid server more performance and that behavior is gone.
Re: [squid-users] How often should I restart Squid?
Folks, I had this weird behavior with squid on a RH9 server using kernel 2.4.22 , 2Gb of memory (the server was DL380 G3 from Compaq/HP), the swap usage was slowly growing forever and I even try disabling it with no harms , but I didn't want do it ...why ? It shouldn't swap ! My squid is set behind a apache + mod_proxy as a web accelerator caching requests for other sites in a reverse proxy configuration. It is very busy ( ~ 160 reqs/s ) . 10 days ago I upgrade to kernel 2.6.6, guess what . NO swap usage anymore ! All the memory is allocated, but ~50 % of it is cached . Upgrading the kernel on my squid servers went pretty good for me so far ... regards Bruno Marcondes
Re: [squid-users] How often should I restart Squid?
On Mon, 21 Jun 2004 03:31:26 +, Lizzy Dizzy [EMAIL PROTECTED] wrote: When squid started using so much swapspace, does it means that there's many obsolete objects inside the RAM that has been forgotten and thus will not be removed/swapped out? What is your memory setting on squid.conf ? Kernel version ?
Re: [squid-users] How often should I restart Squid?
On 22.06 22:47, Hendrik Voigtlnder wrote: With growing I mean the squid process increasing over the time. If I read the FAQ correctly this could be caused e.g. if more and more objects go in the the cache_dirs thus increasing the space needed for the index, i.e. if squid starts with a clean cache_dir. However, my squid is no longer increasing in process size, cache dirs are full the load is the same all day. Snapshot from top (idle squid at night). 21603 proxy 9 0 1017M 1.0G 1180 S 0.0 50.3 1:46 squid It uses roughly 50% of the RAM (machine has 2GB) , the rest is used by other processes and buffers/cache. I just think it is dangerous to disable swap, if one doesn't know how large the squid process will get, i.e. probably larger than the physical memory and this causing the OS to kill processes randomly (I had this problem with java-stuff eating up all memory). I do not think so. The same can happen if you have swap or not - it's always used as virtual memory. Having more VM just delays running machina out of it, which only happens if there's some memory leak in squid, libraries or other applications running on that machine. If your squid is not using more memory for longer time, we can assume there's no leak, and as long as it only uses half of the RAM, you have still 1 GB free for processes etc, so you probably do not need swap at all. What puzzles me is that my machine started to use swap at all as plenty of memory is available, that is why I disabled swap with a perfomance boots as a result. That is feature of VM systems, they store unused data onto swap even if it's not needed. If in any case the mamory use will grow, unused data will not have to be stored on disk because they alredy are, so the system will spare swapping that time. -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Nothing is fool-proof to a talented fool.
Re: [squid-users] How often should I restart Squid?
Matus UHLAR - fantomas wrote: On 22.06 22:47, Hendrik Voigtlnder wrote: With growing I mean the squid process increasing over the time. If I read the FAQ correctly this could be caused e.g. if more and more objects go in the the cache_dirs thus increasing the space needed for the index, i.e. if squid starts with a clean cache_dir. However, my squid is no longer increasing in process size, cache dirs are full the load is the same all day. Snapshot from top (idle squid at night). 21603 proxy 9 0 1017M 1.0G 1180 S 0.0 50.3 1:46 squid It uses roughly 50% of the RAM (machine has 2GB) , the rest is used by other processes and buffers/cache. I just think it is dangerous to disable swap, if one doesn't know how large the squid process will get, i.e. probably larger than the physical memory and this causing the OS to kill processes randomly (I had this problem with java-stuff eating up all memory). I do not think so. The same can happen if you have swap or not - it's always used as virtual memory. Having more VM just delays running machina out of it, True, but this is the point: If you have plenty of swap, there is a much better chance to recognise and fix the problem before a crash. which only happens if there's some memory leak in squid, libraries or other applications running on that machine. What about a squid with a huge index (large cache_dir/lot of cache objects) and a large cache_mem setting? What happen if squids minimum memory requirements exceed the virtual memory available (swap or not?). If your squid is not using more memory for longer time, we can assume there's no leak, Quite sure about that, I use the debian/stable package. I never had a problem with my selfcompiled squid's on solaris as well. and as long as it only uses half of the RAM, you have still 1 GB free for processes etc, so you probably do not need swap at all. That is why I disabled it... What puzzles me is that my machine started to use swap at all as plenty of memory is available, that is why I disabled swap with a perfomance boots as a result. That is feature of VM systems, they store unused data onto swap even if it's not needed. If in any case the mamory use will grow, unused data will not have to be stored on disk because they alredy are, so the system will spare swapping that time. Yes, I remember reading something somewhere about that. Isn't that behaviour tunable somehow?
Re: [squid-users] How often should I restart Squid?
On 21.06 20:03, Hendrik Voigtlnder wrote: Yes, our proxy did use swap until I disabled it :-) 2GB RAM, squid process is stable at roughly 1GB process size, nevertheless the machine started swapping. I never figured out why, but the proxy is running fine without any swapspace. This is dangerous with a squid still growing. growing in what way? is the process size still ~1GB? how much of that size is in memory (size/res values in top)? Using swap (for the squid process) affects performance. yes, but it can increase and decrease performance too, that depends on HOW it's being used. I would reduce cache_mem and probably the size of the cache_dir(s) to avoid this situation. I don't think you need to do this... -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 10 GOTO 10 : REM (C) Bill Gates 1998, All Rights Reserved!
Re: [squid-users] How often should I restart Squid?
Matus UHLAR - fantomas wrote: On 21.06 20:03, Hendrik Voigtlnder wrote: Yes, our proxy did use swap until I disabled it :-) 2GB RAM, squid process is stable at roughly 1GB process size, nevertheless the machine started swapping. I never figured out why, but the proxy is running fine without any swapspace. This is dangerous with a squid still growing. growing in what way? is the process size still ~1GB? how much of that size is in memory (size/res values in top)? With growing I mean the squid process increasing over the time. If I read the FAQ correctly this could be caused e.g. if more and more objects go in the the cache_dirs thus increasing the space needed for the index, i.e. if squid starts with a clean cache_dir. However, my squid is no longer increasing in process size, cache dirs are full the load is the same all day. Snapshot from top (idle squid at night). 21603 proxy 9 0 1017M 1.0G 1180 S 0.0 50.3 1:46 squid It uses roughly 50% of the RAM (machine has 2GB) , the rest is used by other processes and buffers/cache. I just think it is dangerous to disable swap, if one doesn't know how large the squid process will get, i.e. probably larger than the physical memory and this causing the OS to kill processes randomly (I had this problem with java-stuff eating up all memory). What puzzles me is that my machine started to use swap at all as plenty of memory is available, that is why I disabled swap with a perfomance boots as a result. Using swap (for the squid process) affects performance. yes, but it can increase and decrease performance too, that depends on HOW it's being used. Can you explain this a bit more? In my experience and according to http://www.squid-cache.org/Doc/FAQ/FAQ-8.html#ss8.11 I can see only performance impacts. I would reduce cache_mem and probably the size of the cache_dir(s) to avoid this situation. I don't think you need to do this... Sorry, probably I need to rephrase that: I dont think that _I_ have to reduce cache_mem and probably the size of the cache_dir(s), but I would do this in Liz (Lizzy Dizzy) situation, i.e. if a squid eats up all memory and starts to use swap to. (http://www.squid-cache.org/Doc/FAQ/FAQ-8.html#ss8.9) Regards, Hendrik Voigtlnder
Re: [squid-users] How often should I restart Squid?
On 21.06 06:20, Lizzy Dizzy wrote: I see, but according to Duane, http://www.onlamp.com/pub/a/onlamp/2004/02/12/squid.html .Another thing that can help is to simply restart Squid periodically, say, once per week. Over time, something may happen (such as a network outage) that causes Squid to temporarily allocate a large amount of memory. Even though Squid may not be using that memory, it may still be attached to the Squid process. Restarting Squid allows your operating system to truly free up the memory for other uses. Is that true? That should be considered as a bug probably... Maybe it's dependant on memory_pools setting... If a client switches off his computer suddenly, is it equivalent to a network outage? I've read the FAQ, but could'nt ascertain that using the SWAP space is a bad thing. Does your caching servers uses up 250MB of SWAP? 250MB of what swap? I have: cache_mem 256 MB cache_dir ufs /mount/proxy 3 256 256 UP Time:4487960.137 seconds Process Data Segment Size via sbrk(): 896024 KB Memory accounted for: Total accounted: 477263 KB PID USERNAME PRI NICE SIZERES STATE C TIME WCPUCPU COMMAND 328 proxyd 2 0 879M 704M poll 0 605:00 0.00% 0.00% squid the machine has 2GB of memory... -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. The early bird may get the worm, but the second mouse gets the cheese.
Re: [squid-users] How often should I restart Squid?
Thanks everyone. I see, but according to Duane, http://www.onlamp.com/pub/a/onlamp/2004/02/12/squid.html .Another thing that can help is to simply restart Squid periodically, say, once per week. Over time, something may happen (such as a network outage) that causes Squid to temporarily allocate a large amount of memory. Even though Squid may not be using that memory, it may still be attached to the Squid process. Restarting Squid allows your operating system to truly free up the memory for other uses. If a client switches off his computer suddenly, is it equivalent to a network outage? I've read the FAQ, but could'nt ascertain that using the SWAP space is a bad thing. Does your caching servers uses up 250MB of SWAP? Thanks Liz Yes, our proxy did use swap until I disabled it :-) 2GB RAM, squid process is stable at roughly 1GB process size, nevertheless the machine started swapping. I never figured out why, but the proxy is running fine without any swapspace. This is dangerous with a squid still growing. Using swap (for the squid process) affects performance. I would reduce cache_mem and probably the size of the cache_dir(s) to avoid this situation. Regards, Hendrik
Re: [squid-users] How often should I restart Squid?
Hi Lizzy, Check out this part of the FAQ it might help. http://www.squid-cache.org/Doc/FAQ/FAQ-8.html Regards, Richard On Mon, 21 Jun 2004 03:31:26 +, Lizzy Dizzy [EMAIL PROTECTED] wrote: Hi Folks, I've got a pretty busy squid proxy server (approx 50-100 request /sec). Is there any guideline as to how often do we need to restart squid (e.g. shutdown and start squid)? The reason is that I found out that squid uses up my 2.5GB of RAM leaving only 6MB. Also it uses up half of my 500MB swapspace. When squid started using so much swapspace, does it means that there's many obsolete objects inside the RAM that has been forgotten and thus will not be removed/swapped out? Thanks LIZZ _ Get MSN Hotmail alerts on your mobile. http://en-asiasms.mobile.msn.com/ac.aspx?cid=1002
RE: [squid-users] How often should I restart Squid?
Hi Folks, I've got a pretty busy squid proxy server (approx 50-100 request /sec). Is there any guideline as to how often do we need to restart squid (e.g. shutdown and start squid)? The reason is that I found out that squid uses up my 2.5GB of RAM leaving only 6MB. Also it uses up half of my 500MB swapspace. When squid started using so much swapspace, does it means that there's many obsolete objects inside the RAM that has been forgotten and thus will not be removed/swapped out? Provided your SQUID box is adequately sized in memory terms (see the FAQ); you never have to restart SQUID. M.
