Re: VCL config file for specific URLs
In our vcl_recv, we use things like:
if (req.url ~ wp-admin) {
pass;
}
if (req.url ~ /blog/wp-login.php) {
pipe;
}
// Don't cache from some IPs
if (client.ip == 1.2.3.4) {
pass;
}
Is that any help?
Rob
pub crawler wrote:
Thanks for your input Rob.
JPEG's, GIFs, etc. are all fine to cache, as they are very static in
nature in our environment.
Currently we have Varnish setup to cache:
ico|html|htm|bmp|png|gif|jpg|jpeg|swf|css|js|rss|xml
Our issue is our app servers get overwhelmed, become a large
bottleneck and eventually fail under high load. We can add more
servers in horizontal scaling mode, but creates more wasted power,
more machines to maintain, etc. Our need it to address raised site
load mostly due to search spiders that are out of control but
necessary.
So we thought getting Varnish to cache all these dynamic pages would
alleviate load on our application servers.
Essentially, it is fine for Varnish to cache all of our ColdFusion
pages (.cfm), except a handful of pages like these:
http://www.website.com/Template/members/
http://www.website.com/Template/dsp_addreview.cfm/flat/ID=157
etc.
Any idea of how to accomplish the cache exception of these handful of pages?
I'm not sure of the best way to supply a large list of URLs to pipe, but I'd
suggest that you think about turning the logic around. I don't know the
nature of your site, but presumably it's safe to cache all JPEGs and
similar. How much load would be alleviated by caching everything whose
content type is not text/html?
Then, for text/html, is it possible for you to edit your backend site and
add a header such as X-Is-Cacheable: yes in your index.cfm and Review.cfm?
Then, in vcl_fetch, you'd do something like:
if content-type = text/html
if X-Is-Cacheable = yes
cache this
else
don't cache
end if
else
cache this
end if
You might find this approach simpler than writing a big long list of pages
not to cache.
Rob
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: VCL config file for specific URLs
In message 4c3149fb1001090240l53a26f5eg9452abdbdd820...@mail.gmail.com, pub c
rawler writes:
Any idea of how to accomplish the cache exception of these handful of pages?
If you have a short-ish list of URLs, you can simply test for them:
sub vcl_recv {
if (req.url ~ (foo.html|bar.html|baz.html)) {
pass;
}
}
If the list is long-ish, consider having the webserver mark them
as non-cacheable by adding a HTTP header varnish can check for
instead. That way the knowledge about cacheability is maintained
with the content.
Of course, you can also mix the two methods...
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: VCL config file for specific URLs
In our vcl_recv, we use things like:
if (req.url ~ wp-admin) {
pass;
}
if (req.url ~ /blog/wp-login.php) {
pipe;
}
// Don't cache from some IPs
if (client.ip == 1.2.3.4) {
pass;
}
Is that any help?
Rob
pub crawler wrote:
Thanks for your input Rob.
JPEG's, GIFs, etc. are all fine to cache, as they are very static in
nature in our environment.
Currently we have Varnish setup to cache:
ico|html|htm|bmp|png|gif|jpg|jpeg|swf|css|js|rss|xml
Our issue is our app servers get overwhelmed, become a large
bottleneck and eventually fail under high load. We can add more
servers in horizontal scaling mode, but creates more wasted power,
more machines to maintain, etc. Our need it to address raised site
load mostly due to search spiders that are out of control but
necessary.
So we thought getting Varnish to cache all these dynamic pages would
alleviate load on our application servers.
Essentially, it is fine for Varnish to cache all of our ColdFusion
pages (.cfm), except a handful of pages like these:
http://www.website.com/Template/members/
http://www.website.com/Template/dsp_addreview.cfm/flat/ID=157
etc.
Any idea of how to accomplish the cache exception of these handful of pages?
I'm not sure of the best way to supply a large list of URLs to pipe, but I'd
suggest that you think about turning the logic around. I don't know the
nature of your site, but presumably it's safe to cache all JPEGs and
similar. How much load would be alleviated by caching everything whose
content type is not text/html?
Then, for text/html, is it possible for you to edit your backend site and
add a header such as X-Is-Cacheable: yes in your index.cfm and Review.cfm?
Then, in vcl_fetch, you'd do something like:
if content-type = text/html
if X-Is-Cacheable = yes
cache this
else
don't cache
end if
else
cache this
end if
You might find this approach simpler than writing a big long list of pages
not to cache.
Rob
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
VCL config file for specific URLs
Finally back to looking at Varnish for our overwhelmed site. Working on our default.vcl file. Have a simple, perhaps obvious question. We want to put Varnish in front of our application server. There are cookies coming from the app server, but we have told Varnish to ignore them: unset req.http.cookie That works fine, but we want Varnish to only do this on certain pages. More simply and shorter in number we want Varnish to adhere to the cookies only on certain user customizeable pages. For example (cache these in Varnish): http://www.website.com/Template/index.cfm http://www.website.com/Template/Review.cfm/flat/ID= DO NOTE CACHE THESE - pipe to backend: http://www.website.com/Template/dsp_addreview.cfm http://www.website.com/Template/mapinput.cfm Any ideas on how to best go about providing a list of URLs to pipe in our default.vcl file? Thanks! ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
