Re: Still running out of workspaces ? (sess_workspace=551077)
On Thu, Mar 4, 2010 at 3:57 AM, Simon Males s...@sime.net.au wrote: I don't really know how to interpret this, but I see references to esi and session. ESI is used heavily and sess_workspace is 551077. I know of people running with a session workspace of ~5M. And yes, this really needs to be an autotuning setting. Below I have the Varnish startup parameters. Should I just continue to increase the sess_workspace until it stops happening? Yes. -- Per Andreas Buer, CEO, Varnish Software AS Phone: +47 21 54 41 21 / Mobile: +47 958 39 117 / skype: per.buer ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
varnish-cache.org is moving
Hi. We'll be moving varnish-cache.org on monday march the first around 11:00 CET. The site will be unavailable for about 5 minutes while we move it. We'll be getting a newer trac version but other than that there should be no user-noticable changes. At some point we will also move the mailing lists away from projects.linpro.no and onto varnish-cache.org. More on this when we have a date. Please let me know if you experience any problems. -- Per Andreas Buer, CEO, Varnish Software AS Phone: +47 21 54 41 21 / Mobile: +47 958 39 117 / skype: per.buer ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish CLI user feedback, please.
On Wed, Feb 24, 2010 at 7:37 PM, Poul-Henning Kamp p...@phk.freebsd.dkwrote: I'm looking at the CLI/varnishadm stuff right now, and would like some feedback from you guys... Right now (in -trunk) we have these possible CLI configurations: A) no CLI at all. Probably useful for a lot of simple uses. Some CMS might ship Varnish as part of it and there won't a need for CLI. B) CLI on stdin (-d) C) CLI on TELNET (-T) D) CLI on call-back (-M) I really like this one. If we could have an option to let Varnish start without the cache running (like -d) one could picture some sort of service accepting connections from newly started varnishd servers. The service would then configure the caches, provision them with VCL and start them up. The need for messing around with having shell and config files on the caches disappears. -- Per Andreas Buer, CEO, Varnish Software AS Phone: +47 21 54 41 21 / Mobile: +47 958 39 117 / skype: per.buer ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Varnish Software
Hi list. Redpill-Linpro (RL) has been the main sponsor of Varnish for the last three years. RL is a services company, generating most (probably over 95%) of its revenue on selling services and products in its local Scandinavian market. Developing a product for a global market takes a different kind of organization and a different kind of focus. Therefore RL has decided to form a separate company around Varnish: Varnish Software AS. The company will initially consist of Tollef Fog Heen, Kristian Lyngstøl and myself, all moving from Redpill Linpro. Our sales will intially be handled by Redpill Linpro. We'll expand the staff with more people in the months to come. So, what does this mean for Varnish? 1. Varnish continues as a BSD-Licensed Open Source project. 2. Varnish users worldwide will be able to get competent commercial services, both contracting and service contracts, from a company who specializes in Varnish and high performance web delivery. 3. We will be able to spend more time and money on developing Varnish. Our goal is to make Varnish a de-facto standard for high performance web sites. We're not quite there yet - but we will be. Some of Redpill Linpros service agreement customers will be transferred to the new entity and some will continue to be handled directly by R-L. We are hoping to sign up other partners to cover the whole world. Don't hesitate getting directly in touch with me if you want to discuss partnership, employment, a support agreement or if you are just plain curious. Every detail isn't ironed out yet and more information will be made available later. Regards. Per. -- Per Andreas Buer CEO, Varnish Software AS Phone: +47 21 54 41 21 / Mobile: +47 958 39 117 / skype: perbu ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: flushing cache doesn't seem to make varnish know about less
Timothy Ball wrote: a programming error caused varnish to think there were billions of pages it had to know about. bug is quashed but varnish doesn't seem to know (..) i tried doing url.purge .* and url.purge . to no avail. url.purge .* will only invalidate the pages. They are not actually evicted from cache. HTTP PURGE or TTL timeout are the only ways to actually drop a page from cache, except restarting of course. how do i make this thing forget? I guess the easiest would be a restart. -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: 2.1 plans
Jeff Anderson wrote: I'd like to see individual object request statistics and a method to prefetch objects from the backend that are most frequently requested. Perhaps also a way to prioritize objects into cache tiers based on frequency of requests. So, for example, highly requested objects are maintained in RAM and less frequently requested objects are cached to disk. Your operating system already does this today with Varnish. Squid tries to maintain a two tier cache hierarchy without success. If persistent storage is on its way maybe a method to assign priority to large disk cache volumes versus memory regions. Noted. It might be nice to have a distributed and/or tiered cache model where a single master has a very large cache and potentially very long grace ability where objects can exist even if stale. That master in turn could host frontend caches that communicate efficiently to the master cache and also have a tiered internal object priority. I believe most of this can be achieved today. Stale objects will hopefully reach the 2.0 series before the 2.1 revolutions - at least as a patch, I hope. Thanks, --Jeff On Jan 8, 2009, at 2:29 AM, Tollef Fog Heen wrote: Hi, a short while before Christmas, I wrote up a small document pointing to what I would like to get into 2.1 and when I'd like milestones to happen. This is a suggestion, I'm open to ideas and comments on both feature set as well as if my guesstimates for dates is completely off: Varnish 2.1 release plan The theme for Varnish 2.1 is scalability, particularly trying to address the needs of sites like finn.no which has a lot of objects and where priming the cache takes a long time, leading to long periods of higher load on the backend servers. The main feature is persistent storage, see http://varnish.projects.linpro.no/wiki/ArchitecturePersistentStorage for design notes. Another important scalability feature is a new lockless hash algorithm which scales much better than the current one. Poul-Henning already has an implementation of this in the tree, but it's still fresh. Minor features which would be nice to get in are: * Web UI, showing pretty graphs as well as allowing easy configuration of a cluster of Varnish machines. * Expiry randomisation. This reduces the lemmings effect where you end up with a many objects with almost the same TTL (typically on startup) which then expire at the same time. The feature will allow you to set the TTL to plus/minus X %. * Dynamic, user-defined counters that can be read and written from VCL * Forced purges, where a thread walks the list of purged objects and removes them. The schedule Alphas: - 2009-01-15: New hash algorithm working - 2009-02-15: Web UI - 2009-03-15: Persistent storage Beta: - 2009-04-01: Feature complete Release - 2009-05-20: Release candidate - 2009-05-01: No release critical bugs left - 2009-05-10: Release -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc --Jeff j...@funnyordie.com ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Example VCL for backend check
Hi Alex.
Good to hear you got it working.:-) I written a page about load
balancing a few days ago and I've added your vcl_recv there. It's
located at http://varnish.projects.linpro.no/wiki/LoadBalancing - linked
from http://varnish.projects.linpro.no/wiki/VarnishFeatures - it might
need some proofreading if anyone got some spare time.
I'm not aware of any way to monitor pageload time from varnish. I guess
the information is available in varnish so it might not be to hard to
expose this in VCL. At the moment its not on the list, however.
Per.
Alex Davies wrote:
Hi Per,
Thank-you for your help? With a minor change, your configuration worked
extremely well - is there a wiki for Varnish that I can post it? I've
written out a full, working, configuration for a two-host director below
which might be of use in the VCL examples page.
Another question - is there an easy way to monitor the average pageload
time that varnish experiences when contacting the backends?
Many thanks for your help,
Alex
--
backend wwxx {
.host = xx.xx.xx.xx;
.port = 80;
.probe = {
.url = /;
.timeout = 34 ms;
.interval = 1s;
.window = 10;
.threshold = 8;
}
}
backend wwxx {
.host = xx.xx.xx.xx;
.port = 80;
.probe = {
.url = /;
.timeout = 34 ms;
.interval = 1s;
.window = 10;
.threshold = 8;
}
}
director director random {
{
.backend = wwxx;
.weight = 100;
}
{
.backend = wwxx;
.weight = 100;
}
}
sub vcl_recv {
if (req.http.host ~ ^(www.)?mysite.com http://mysite.com$) {
set req.backend = director;
}
}
On Thu, Dec 11, 2008 at 6:03 PM, Per Buer pe...@linpro.no
mailto:pe...@linpro.no wrote:
Hi Alex,
Alex Davies wrote:
I'm figuring that I need something like
# web1
backend default {
backends must have uniqe names so you might use:
backend web1 {
.host = xxx.xxx;
.port = 80;
.probe = {
.url = /;
.timeout = 34 ms;
.interval = 1s;
.window = 10;
.threshold = 8;
}
}
# web2
backend default {
backend web2{
.host = xxx.xxx;
.port = 80;
.probe = {
.url = /;
.timeout = 34 ms;
.interval = 1s;
.window = 10;
.threshold = 8;
}
}
And then something to glue all the backends together, but I see many
different options in the manuals! I'd appreciate a pointer in the
right direction if anyone could spare one. The vcl man page isnt
entirely obvious to me!
Use a director, like this:
director www-director round-robin {
{
.backend = web1;
.weight = 100;
}
{
.backend = web2;
.weight = 200;
}
}
--
Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
Telefon: 21 54 41 21 - Mobil: 958 39 117
http://linpro.no/ | http://redpill.se/
___
varnish-misc mailing list
varnish-misc@projects.linpro.no mailto:varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
--
Alex Davies
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the sender immediately by e-mail and delete this e-mail permanently.
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
--
Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
Telefon: 21 54 41 21 - Mobil: 958 39 117
http://linpro.no/ | http://redpill.se/
signature.asc
Description: OpenPGP digital signature
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: What to do when the backend sends no caching information?
Hi, Alecs Henry wrote: Well, turns out varnish may be setting the ttl all right (I do it in vcl_fetch, but frankly there really is no way to see if it is set correctly or not! or is there?) but the page always gets a PASS, never a HIT. Age: is set to 0 by varnish. The client probably sends a cookie. When varnish sees a cookie it does a pass. Please see the FAQ on notes on how to deal with this. Per. -- http://linpro.no/ | http://redpill.se/ signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Example VCL for backend check
Hi Alex,
Alex Davies wrote:
I'm figuring that I need something like
# web1
backend default {
backends must have uniqe names so you might use:
backend web1 {
.host = xxx.xxx;
.port = 80;
.probe = {
.url = /;
.timeout = 34 ms;
.interval = 1s;
.window = 10;
.threshold = 8;
}
}
# web2
backend default {
backend web2{
.host = xxx.xxx;
.port = 80;
.probe = {
.url = /;
.timeout = 34 ms;
.interval = 1s;
.window = 10;
.threshold = 8;
}
}
And then something to glue all the backends together, but I see many
different options in the manuals! I'd appreciate a pointer in the
right direction if anyone could spare one. The vcl man page isnt
entirely obvious to me!
Use a director, like this:
director www-director round-robin {
{
.backend = web1;
.weight = 100;
}
{
.backend = web2;
.weight = 200;
}
}
--
Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
Telefon: 21 54 41 21 - Mobil: 958 39 117
http://linpro.no/ | http://redpill.se/
signature.asc
Description: OpenPGP digital signature
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish performance tests
Rebert Luc wrote: Hello, In our studies we have a project which consists in testing the performance of Varnish in order to make a comparative with and without the proxy cache. Does anyone know which utilities to employ ? (knowing that the aim is to justify our tests) You could have a look at varnishreplay to replay earlier recorded varnishlogs. If you also want to synthesize traffic you could also check out siege and curl-loader. If you have a high hitrate in your cache you'll most likely end up benchmarking the tools rather then varnish. I think I measured siege to be ~3x slower then varnish. -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: varnish2.0.2 on Suse 10.3
Tollef Fog Heen skrev: ]] Paras Fadte | child (28673) Started | Pushing vcls failed: Internal error: No VCL_conf symbol It seems to fail to load the configuration, probably because the .so with the configuration ends up stripped. Why that happens, I have no idea. It seems to be a recurring issue on OpenSUSE. There have been earlier reports on the same issue. -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Using req/obj.grace to serve stale objects when backend fails
Hi Jeff. Jeff Anderson skrev: I've experimented with req/obj.grace set to a few hours so a site can be served even if the backend fails. For example, with the backend down and when req/obj.grace is set to several hours I can open a new browser and get a 503 if I try to open a known cached (and graced) page. You're right. It shold be fixed. There is now a ticket (#369) on the matter. -- http://linpro.no/ | http://redpill.se/ signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: TCP_HIT header
Hi.
As I said. Add two different headers. One you add in vcl_hit and one
(preferably a different one) in vcl_miss. You need no variables. The
code in vcl_hit will be run for a hit and vcl_miss will be run for a miss.
Check out the getting started guide, the FAQ and the VCL-page on Wiki if
you seek documentation. Thats all there is, at the moment.
Per.
Alecs Henry skrev:
Hi Per,
Thanks for the reply!
The issue here is not how to add the header, that is OK, I can do it
just fine (even added the Foo: bar header just for the fun of it!). The
problem is WHAT variable I can use that contains that information (MISS
or HIT).
Is there more documentation on the variables available than what is in
vcl(7)?
Or anywhere else for that matter.
Thanks,
Alecs
On Wed, Nov 5, 2008 at 3:10 PM, Per Buer [EMAIL PROTECTED]
mailto:[EMAIL PROTECTED] wrote:
Alecs Henry skrev:
Hi guys,
Is there a variable that I can print on the response header that will
give me the cache lookup result such as TCP_HIT or TCP_MISS?
I guess you can add the relevant header in vcl_hit and vcl_miss
See the FAQ:
http://varnish.projects.linpro.no/wiki/FAQ#HowdoIaddaHTTPheader
Just add a
sub vcl_hit {
# add code from faq here
}
--
http://linpro.no/ | http://redpill.se/
--
Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
Telefon: 21 54 41 21 - Mobil: 958 39 117
http://linpro.no/ | http://redpill.se/
signature.asc
Description: OpenPGP digital signature
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Guru Meditation error
Could you supply the complete varnishstat output? I _guess_ this is a misbehaving Plone installation. Per. Mhairi Marshall skrev: Hi, I have upgraded to varnish 2.0.1 and got the error again. 0 0.00 0.00 Backend connections not attempted 0 0.00 0.00 Backend connections too many 6 0.00 0.00 Backend connections failures 244 0.00 0.00 Backend connections reuses 246 0.00 0.00 Backend connections recycles 0 0.00 0.00 Backend connections unused It seems to be worse in the morning when the website hasn't been used overnight. Our network guys say it is an application problem as the application needs to send keep alives. Thanks Mhairi -Original Message- From: Per Buer [mailto:[EMAIL PROTECTED] Sent: Wednesday, 22 October 2008 9:03 PM To: Mhairi Marshall Cc: varnish-misc@projects.linpro.no Subject: Re: Guru Meditation error Mhairi Marshall skrev: I have been getting the Guru Meditation when using varnish (version 1.1.2) cache for Plone sites. I have seen from the mailing list that some people have been having the same problem but haven't seen any solutions posted. Is there any configuration for varnish or apache that will stop this error from happening? Its hard to say without any more information. Anyway, you should really upgrade to Varnish 2.0.1 (or 2.0.2 if its out). Varnish 2 is much more robust. This might be because Plone sometimes misbehavies - but try an upgrade and get back to us if you're still having trouble. -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Guru Meditation error
Mhairi Marshall skrev: I have been getting the Guru Meditation when using varnish (version 1.1.2) cache for Plone sites. I have seen from the mailing list that some people have been having the same problem but haven’t seen any solutions posted. Is there any configuration for varnish or apache that will stop this error from happening? Its hard to say without any more information. Anyway, you should really upgrade to Varnish 2.0.1 (or 2.0.2 if its out). Varnish 2 is much more robust. This might be because Plone sometimes misbehavies - but try an upgrade and get back to us if you're still having trouble. -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Filtering by specific cookie
Mikkel Høgh skrev:
In http://varnish.projects.linpro.no/wiki/VCLExampleRemovingSomeCookies,
the example shows removing cookies with regular expressions via the
regsub function which I can't seem to find documentation on.
Is there a similar function just for checking whether there's a match,
or even better, a way to check this that doesn't involve regular
expressions?
in vcl_revc you can do stuff like:
if (req.http.cookie ~ foo.*bar|bar.*foo) {
lookup;
}
--
Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
Telefon: 21 54 41 21 - Mobil: 958 39 117
http://linpro.no/ | http://redpill.se/
signature.asc
Description: OpenPGP digital signature
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Blades
Andre Galvani skrev: Hello, someone has had experience running varnish on blades(DELL HW for example)? Yes. We run a lot of Varnish on blades. Runs smoothly. :-) -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish cache lost after restarted
Leo Wang skrev: When I restart varnish, the cache file is still there but it seems that all cache contents are lost. All objects will have to be re-fetched from backends even they are not expired. Is it a designed behavior? Maybe not designed, but definitively expected. A persistent storage engine is more or less planned for 2.1. -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Ip Address
Jack Tuhman skrev: We have a very old webapp that has an embedded server. We have put varnish in front of it to help with load while we rewrite it. However, it is detecting the IP address of the varnish box rather then the client. I know that in apache, we can fix this by looking at HTTP_X_FORWARDED_FOR, however, our old app does not let us do that. Is there anything we can do in varnish to pass the real ip in the REMOTE_ADDR server var? The REMOTE_ADDR server variable is set locally on the web server. So, unless you can write some custom code to override REMOTE_ADDR you are out of luck. :-( Per. signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: varnish inside openvz: memory usage issue
Isaac Grant skrev: (..) I (think I) understand the varnish's design, but should varnish eat ALL the memory like that ? Varnish uses the VM to differentiate between what's cached in memory and what is put on disk. If the VM doesn't have any other use for the memory it will give it to varnish. The problem is that you are running VM inside a box (the openvz container) which artificially limits the memory usage. If this was a sane VM it would probably page out a lot of Varnish cache and not run out. I guess this is a bug or a weakness of openvz, first and foremost. You could work around the problem by limiting the cache size. I don't think you will encounter the same problem with Xen or any other virtualization solution where you have a separate VM for your VPS. Per. signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Waiting list?
Jon Drukman skrev: 92 Debugc on waiting list on obj 1629712739 0 StatAddr 64.81.61.44 0 nan 16 16 0 0 3 5483 637585 That's it. After the waiting list message the request just hangs indefinitely. You might want to have a look at session 1629712739 - earlier in the logs. I'm not sure how many people are running varnish with the origin server available on a flaky WAN connection. There might be issues. Per. signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Question about threads
Erik Torlen skrev: Im running 32bit. But I think that I have succeded creating more then 238 threads before on another system with the same setup. Anyway, 64bit might be the thing to have... Maybe the init script should issue a warning that a 32bit arch is only usable as a test enviroment. Per. signature.asc Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
