Re: Varnish shutting down under heavy load
Poul-Henning Kamp wrote: I've seen the automatic restart of the worker process fail here during my testing, and I suspect you hit the same thing. I have it on my list of things to investigate RSN. Do you have any corefiles from varnishd by any chance ? I just set ulimit -c unlimited and started varnish again. Regards. A.S ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish shutting down under heavy load
Anup Shukla wrote: Poul-Henning Kamp wrote: I've seen the automatic restart of the worker process fail here during my testing, and I suspect you hit the same thing. I have it on my list of things to investigate RSN. Do you have any corefiles from varnishd by any chance ? I just set ulimit -c unlimited and started varnish again. Regards. Did not take it long to go down again. But i am not able to find any core dumps. Some help as to how to get one will be appreciated. The logs said 843 TxHeader c X-Varnish: 1617761482 1617756991 843 TxHeader c Age: 75 843 TxHeader c Via: 1.1 varnish 843 ReqEnd c 1617761482 1183726324.435635000 1183726324.435695000 4.93416 0.19000 0.41000 0 StatAddr 124.64.56.126 0 31 11 15 0 0 1 4243 180969 1017 SessionOpen c 58.214.216.252 3364 0 CLIRd vcl.load boot /tmp/vcl.XXnkvV6y 0 CLIWr 0 200 Loaded /tmp/vcl.XXnkvV6y as boot 0 Error CLI read 0 (errno=4) A.S ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Problems with revision 1643.
Hi All,
A few things are causing trouble in r1463
Varnish always uses uid nobody and gid nogroup.
i did this:
#useradd varnish
#varnishd -u varnish -g varnish
But, it still complained about group: nogroup not being present.
I added a group nogroup, just to keep varnish happy.
Varnish then complained about my VCL configuration in vcl_fetch()
snip
if (resp.http.Content-Type ~ text/html) {
pass;
}
/snip
Is this something wrong?
The version that i am currently using (1.0.4 20 May 2007) never gave an
error about resp.http.Content-Type.
Thank you.
Regards
A.S
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Failure cenarios?
James Quacinella wrote: I use monit for monitoring programs. Here is a snippet I had used when monitoring a varnish install (too bad it never went into production; change values to you liking / environment): A Billion thanks to you :) I have been looking for a tool that could do this for me. Regards A.S ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Problem with varnish and caching
Manuel Amador (Rudd-O) wrote: site is cached according to Varnish default policies. You have not provided a single counterexample or a single snippet of VCL that could solve the problems I have, or a single snippet of VCL that you guys are actually using on production servers. man vcl does provide an example about how to cache even if Cookies are present. Did i miss something? Regards A.S ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Best Practices - Suggestions Request
Hi All. Apologies for not being able to respond to my own problems ;) I would personally like to give the latest SVN version a try. I had encountered a situation where the /tmp was mounted noexec and varnish failed to load. This made me aware that Varnish uses /tmp to keep the compiled VCL. I have removed the noexec option and its fine now.. would love to have a configuration option to set the directory though :) . As for the crashes, its still a mystery. The system does not swap and everything seems to be quite fine. Must be something else which is affecting Varnish. There is one possibility though. There is a lot of old unorganized php code on the servers that i have never had time to inspect. Its working so i did not need to tinker with it. I assume that some one has used the /tmp to cache query responses and might be cleaning up the entire /tmp to flush the caches. Sounds a bit stupid, but a very possible scenario in my case. Will need some time to find if that is really the case. Thanks to DES for pointing this /tmp scenario (in his reply to Tron Michelson) Regards A.S ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: setting varnish storage size
jean-marc pouchoulon wrote: Hi , I'm a bit confused about /var/lib/varnish/varnish_storage.bin I use varnish 1.0.0.4 under linux fedora 6 ( using a vserver ). I set in /etc/sysconfig varnish VARNISH_STORAGE_SIZE=2G and the size of the file still remains to one GIGA -rw--- 1 root root 1,0G jun 29 19:27 /var/lib/varnish/varnish_storage.bin setting VARNISH_STORAGE_SIZE=2000G /var/lib/varnish/varnish_storage.bin setting 3000G I get -rw--- 1 root root 1,5G jun 29 19:29 /var/lib/varnish/varnish_storage.bin mm what I don't understand ? ( the % parameter ?) thanks Are you using 32-bit hardware? If yes, there lies the problem. I had similar issues when testing on my local 32-bit PC. However once i installed on our actual servers (64-bit), the 1G limit went away. Have never tried setting the storage file to sizes in thousands of G though. Also, as far as i know, Linux will not support files bigger than 4G unless the support is added to the kernel. Not sure if Fedora Core 6 has the patch applied. I cannot be very sure of this though. Regards A.S ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Best Practices - Suggestions Request
Anup Shukla wrote: Dag-Erling Smørgrav wrote: Still, only the child should die, the parent should automatically restart it. He says he has to restart it manually, which worries me... Anup, what version are you running? DES I am running version 1.0.4 Is that too old to use? If so, i will update it immediately. Sorry for missing out on the release date. It is version 1.0.4 released on May 20, 2007. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Best Practices - Suggestions Request
Hi All, We have been running varnish for over a month on our production servers now. We are using the default configuration with a small change, that is to cache request with Cookies. We have a number of web servers behind a load balancer, each web server running varnish + apache. I understand that running both services on the same server is probably a bad idea, but due to time constraints i had no option but to go ahead with the current setup. The problem is, the varnish process dies every 7 - 10 days, and i have to manually restart it. One solution would be to write a script which monitors varnishd and starts it if not running. I, however, would like to understand Why it dies... I have been monitoring the servers for long times at a stretch and it seems that a flood of requests for big files (100+ MB) seems to get varnishd down. I cannot be very sure about this though. Is it really a bad idea to run varnishd on the same server with apache? Another thing is, is it possible to configure varnish such that it adds a custom header to the response (for example, the ip of the server which processed the request or any custom value). This would greatly assist in knowing which server served the request. Is it possible to make varnish refuse connections from the clients it it detects that the backend has stopped responding and start servicing again the moment it detects that the backend is up? Sorry, if i have asked too many questions at one go. Any help/pointers would be greatly appreciated. Regards A.S. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Best Practices - Suggestions Request
Poul-Henning Kamp wrote: In message [EMAIL PROTECTED], Anup Shukla writes: I understand that running both services on the same server is probably a bad idea, but due to time constraints i had no option but to go ahead with the current setup. It is not an obviously bad idea, and in many cases it is likely to work quite well. If it works for you, be happy about the saved hardware :-) That makes me feel a lot better now. Thank you. :) It's not much to go from, but here are some ideas: 1. Do you have enough storage space ? By default Varnish takes a fixed fraction of the free space in /tmp, that may not be enough. Use the -s argument to specify a different directory and/or how much space you want to use. I have put the cache file (or whatever its called as) under /cache/varnish the parameters are -s file,/cache/varnish/varnish_storage.bin,2G Disk space should not be a problem as there is plenty available. Initially i had kept the size to 8G, but changed it 2G later. I doubt if that has any affect apart from being able to store not more than 2G of cached content. The only reason i changed it to 2G was that, i noticed the size of varnishd process kept on increasing. Now, i am not a Linux expert so did not have the knowledge about the whole idea of using disk space as RAM. However a bit of Google search cleared up that my fears were baseless. Or is it that there is a relation between the process size and the on-disk cache file? ( I hope this is not out-of-place) Did not change it back to 8G though, it works fine for me with the way its configured. ;) I will keep an eye on the process to see if i can find anything specific to relate to the problem of Varnish dying in 7-10 days. Do i need to check for something specific? Sorry, if i have asked too many questions at one go. You're welcome :-) Much appreciated. Thanks again. :) Regards A.S ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Best Practices - Suggestions Request
Poul-Henning Kamp wrote: In message [EMAIL PROTECTED], Anup Shukla writes: Poul-Henning Kamp wrote: I have put the cache file (or whatever its called as) under /cache/varnish the parameters are -s file,/cache/varnish/varnish_storage.bin,2G Disk space should not be a problem as there is plenty available. Initially i had kept the size to 8G, but changed it 2G later. I doubt if that has any affect apart from being able to store not more than 2G of cached content. Well, that could be your problem, right now Varnish doesn't deal well with running out of storage, (DES is working on that right now) Okay. Then i just increase the storage size to a large but acceptable number. Thank you for the help. Will surely update you if i still have the crashes going on. Thank you again. Regards A.S ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Newbie : Static content caching
Hello all, I have been using Varnish with its default configuration for quite sometime now. Absolutely nothing has changed except for the backed servers port number (set to 8000). Needless to say that it has been performing brilliantly and has reduced the load on our webservers considerably. Just noticed something strange though. Before i explain further, please let me clarify that i am not an expert in caching by any standard, so this happens to be more of a newbie question. I am running varnish and the backend apache on the same server. The backend listens on the loopback address while varnish listens on the interface facing the load balancer. If i request, say for example, a static jpeg, i assume that the first request will be served by the webserver via varnish, and all later requests would be served by varnish, without asking for it from the webserver, that is till the object exceeds its lifetime in the cache. What i observe is this: the request for the jpeg is sent to the webserver everytime, the webserver send the entire content to varnish, and varnish sends a 304 Not Modified to the client browser. Is this the expected behaviour? Do i need to do some VCL'ing to get it work the way i expect? Any help or pointers will be very much appreciated. Thank you. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Newbie : Static content caching
Hello all, Please accept my apologies for posting this without complete understanding about varnish. The reason for the static content not being cached was cookies. Everything is well and good after a slight tinkering of the configuration file. I, again, apologies for not being responsible enough before posting. Just that, in this case, the light flickered, almost immediately after i hit the send button. Thank you. Anup Shukla wrote: Hello all, I have been using Varnish with its default configuration for quite sometime now. Absolutely nothing has changed except for the backed servers port number (set to 8000). Needless to say that it has been performing brilliantly and has reduced the load on our webservers considerably. Just noticed something strange though. Before i explain further, please let me clarify that i am not an expert in caching by any standard, so this happens to be more of a newbie question. I am running varnish and the backend apache on the same server. The backend listens on the loopback address while varnish listens on the interface facing the load balancer. If i request, say for example, a static jpeg, i assume that the first request will be served by the webserver via varnish, and all later requests would be served by varnish, without asking for it from the webserver, that is till the object exceeds its lifetime in the cache. What i observe is this: the request for the jpeg is sent to the webserver everytime, the webserver send the entire content to varnish, and varnish sends a 304 Not Modified to the client browser. Is this the expected behaviour? Do i need to do some VCL'ing to get it work the way i expect? Any help or pointers will be very much appreciated. Thank you. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Newbie : Static content caching
Hi Dag-Erling Smørgrav, Thank you for your response. It was the cookies. Forcing varnish to cache requests even if cookies are present does the trick. Thank you again. Dag-Erling Smørgrav wrote: Anup Shukla [EMAIL PROTECTED] writes: If i request, say for example, a static jpeg, i assume that the first request will be served by the webserver via varnish, and all later requests would be served by varnish, without asking for it from the webserver, that is till the object exceeds its lifetime in the cache. What i observe is this: the request for the jpeg is sent to the webserver everytime, the webserver send the entire content to varnish, and varnish sends a 304 Not Modified to the client browser. Most likely, the backend does not set an expiry time on the requested image, so Varnish uses the default TTL (normally 120s). Another possibility is that your site uses cookies; by default, Varnish will not serve requests that come with a cookie from its cache. I can provide a more detailed (and correct) explanation if you send me a raw varnish log off-list. DES ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
