Re: VCL config file for specific URLs
In message 4c3149fb1001090240l53a26f5eg9452abdbdd820...@mail.gmail.com, pub c
rawler writes:
Any idea of how to accomplish the cache exception of these handful of pages?
If you have a short-ish list of URLs, you can simply test for them:
sub vcl_recv {
if (req.url ~ (foo.html|bar.html|baz.html)) {
pass;
}
}
If the list is long-ish, consider having the webserver mark them
as non-cacheable by adding a HTTP header varnish can check for
instead. That way the knowledge about cacheability is maintained
with the content.
Of course, you can also mix the two methods...
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: VCL config file for specific URLs
In our vcl_recv, we use things like:
if (req.url ~ wp-admin) {
pass;
}
if (req.url ~ /blog/wp-login.php) {
pipe;
}
// Don't cache from some IPs
if (client.ip == 1.2.3.4) {
pass;
}
Is that any help?
Rob
pub crawler wrote:
Thanks for your input Rob.
JPEG's, GIFs, etc. are all fine to cache, as they are very static in
nature in our environment.
Currently we have Varnish setup to cache:
ico|html|htm|bmp|png|gif|jpg|jpeg|swf|css|js|rss|xml
Our issue is our app servers get overwhelmed, become a large
bottleneck and eventually fail under high load. We can add more
servers in horizontal scaling mode, but creates more wasted power,
more machines to maintain, etc. Our need it to address raised site
load mostly due to search spiders that are out of control but
necessary.
So we thought getting Varnish to cache all these dynamic pages would
alleviate load on our application servers.
Essentially, it is fine for Varnish to cache all of our ColdFusion
pages (.cfm), except a handful of pages like these:
http://www.website.com/Template/members/
http://www.website.com/Template/dsp_addreview.cfm/flat/ID=157
etc.
Any idea of how to accomplish the cache exception of these handful of pages?
I'm not sure of the best way to supply a large list of URLs to pipe, but I'd
suggest that you think about turning the logic around. I don't know the
nature of your site, but presumably it's safe to cache all JPEGs and
similar. How much load would be alleviated by caching everything whose
content type is not text/html?
Then, for text/html, is it possible for you to edit your backend site and
add a header such as X-Is-Cacheable: yes in your index.cfm and Review.cfm?
Then, in vcl_fetch, you'd do something like:
if content-type = text/html
if X-Is-Cacheable = yes
cache this
else
don't cache
end if
else
cache this
end if
You might find this approach simpler than writing a big long list of pages
not to cache.
Rob
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
varnish suddenly restarting / flushing itself after several hours?
Hello, After setting up varnish and starting it I looked at the statistics using varnsihstat. The timer in the left uppper corner seems to be the uptime since it last started or was flushed. When I loaded some pages, I can see the cache is working But I notice that once every so often the cache seems to either flush itself or restart. During this 2-3 seconds that this happens I can not load any pages. Is there a default 'cash flush' value that makes this (for me undesirable) behavior happen? I added the example described in http://varnish.projects.linpro.no/wiki/VCLExampleLongerCaching to my vcl file, but if the cash flushes itself a couple of times per day then it defeats the purpose. I am using varnish-2.0.5-1.el5 on centos5 and use the default.vcl file with the above longer caching example added to it and ignore swf and pdf files. I start varnish with 40 MB of cache and test this with a few html pages and jpgs (less then 15 MB in size). During tests normally not more than 50% of the cache is used. Besides this issue varnish seems to work almost right out of the box. Frank. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Caching pages with URL parameters
Managed to work though my earlier Varnish issues with help from the
list. Thank you!
Have another likely obvious question.
Most of our pages use a flat - fake non-appearing to be dynamic
format. Like this:
http://www.website.com/Template/dsp_restaurant_zoom.cfm/flat/ID=460401
For some reason we can't get Varnish to cache these pages.
In sub vcl_recv I have this:
if (req.url ~ dsp_restaurant_zoom.cfm) {
unset req.http.cookie;
lookup;
}
Anyone have any ideas of how to get these pages cached?
Thank you!
-Paul
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Varnish poisoned cache avoidance
We have some ban / block logic in our application server behind Varnish. For instance, when we have a comment spammer or other repetitive troublemaker messing with our applications we ban their IP in our application server. A person or bot returning after being blocked will still reach our app server, but it just returns a page that says BANNED. We had such a banned IP request a page and subsequently I requested the same page and was given the BANNED message as it was sitting in Varnish cache - even though my IP is not banned. My question here is how best to prevent this and what sort of workaround other folks have for this? I've considered banning at our firewall level, but it's too time consuming to do so and the block lists are so long that it really causes the firewall to take forever to restart from cold reboot. Originally I had blocked at the firewall, so I've been down that road. Any input would be greatly appreciated... -Paul ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: varnish suddenly restarting / flushing itself after several hours?
The parent will restart the child if it becomes unresponsive to pings. This can happen for multiple reasons. Do you have historical metrics for the server that you can examine to see if load goes up or idle cpu goes down or if memory is consumed? (Sorry for the top post, friggin mobile device) On Jan 9, 2010 5:34 AM, Frank van Lingen fr...@vanlingen.name wrote: Hello, After setting up varnish and starting it I looked at the statistics using varnsihstat. The timer in the left uppper corner seems to be the uptime since it last started or was flushed. When I loaded some pages, I can see the cache is working But I notice that once every so often the cache seems to either flush itself or restart. During this 2-3 seconds that this happens I can not load any pages. Is there a default 'cash flush' value that makes this (for me undesirable) behavior happen? I added the example described in http://varnish.projects.linpro.no/wiki/VCLExampleLongerCaching to my vcl file, but if the cash flushes itself a couple of times per day then it defeats the purpose. I am using varnish-2.0.5-1.el5 on centos5 and use the default.vcl file with the above longer caching example added to it and ignore swf and pdf files. I start varnish with 40 MB of cache and test this with a few html pages and jpgs (less then 15 MB in size). During tests normally not more than 50% of the cache is used. Besides this issue varnish seems to work almost right out of the box. Frank. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
