Re: Survey; how do you use Varnish?
Hi, I precise: I don't work for Linpro. ;) For some big project, I'm planing to deploy Varnish. So I would be interested if you could precise the Operating system you're using to run Varnish and the hardware type (CPU, memory, HD, NICs). Does some of you run it in a virtualized environment? (Xen, kvm, vmware). thanks a lot On Fri, Jan 29, 2010 at 3:48 PM, Per Andreas Buer wrote: > Hi list. > > I'm working for Redpill Linpro, you might have heard of us - we're the main > sponsor of Varnish development. We're a bit curious about how Varnish is > used, what features are used and what is missing. What does a typical > installation look like? The information you would choose to reveal to me > would be aggregated and deleted and I promise you I won't use it for any > sales activities or harass you in any way. We will pubish the result on this > list if the feedback is significant. If you have the time and would like to > help us please take some time and answer the questions in a direct mail to > me. Thanks. > > 1) How many servers do you have running Varnish? > > 2) What sort of total load are you having? Mbit/s or hits per second are > preferred metrics. > > 3) What sort of site is it? > *) Online media > *) Cooperate website (ibm.com or similar) > *) Retail > *) Educational > *) Social website > > 4) Do you use ESI? > > 5) What features are you missing from Varnish. Max three features, > prioritized. Please refer to > http://varnish-cache.org/wiki/PostTwoShoppingList for features. > > > > -- > Per Andreas Buer > Redpill Linpro Group - Changing the Game > > Mobile +47 958 39 117 / Phone: +47 21 54 41 21 > ___ > varnish-misc mailing list > varnish-misc@projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
1) One active server. We have another one as hot standby. 2) 50Mbit, 200 requests/second max. Most of the time it's 10Mbit, 40 requests/second which isn't much. 3) Internet touroperator. 4) Nope 5) Automatic refreshing of data without having the endusers have to wait for the response. The reason we use varnish most is because our website has complex, timeconsuming queries to backend systems. The answers to these queries do vary several times per day but are still cachable. Of course varnish also helps te bring down the load on those backend systems but the main use is that varnish gives the endusers a lightning fast prerendered interactive experience which is a paradox. We like working paradoxes. Something like 'refresh pages after object.prefetch seconds if at least someone requested that object the last object.ttl seconds' where object.ttl is larger then object.refresh. So an object might be prefetched even a couple of times without anyone being interested but will be removed from the cache eventually after object.ttl has expired. Regards, Martin Boer Per Andreas Buer wrote: > Hi list. > > I'm working for Redpill Linpro, you might have heard of us - we're the main > sponsor of Varnish development. We're a bit curious about how Varnish is > used, what features are used and what is missing. What does a typical > installation look like? The information you would choose to reveal to me > would be aggregated and deleted and I promise you I won't use it for any > sales activities or harass you in any way. We will pubish the result on this > list if the feedback is significant. If you have the time and would like to > help us please take some time and answer the questions in a direct mail to > me. Thanks. > > 1) How many servers do you have running Varnish? > > 2) What sort of total load are you having? Mbit/s or hits per second are > preferred metrics. > > 3) What sort of site is it? > *) Online media > *) Cooperate website (ibm.com or similar) > *) Retail > *) Educational > *) Social website > > 4) Do you use ESI? > > 5) What features are you missing from Varnish. Max three features, > prioritized. Please refer to > http://varnish-cache.org/wiki/PostTwoShoppingList for features. > > > > ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
1) 15 2) Arround 10k/s results in 500-600Mbit. 3) Social Network website 4) No 5) Large dataset performance improvements, High traffic performance improvements rr ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
1) ~20 (Mixture of production and development environments, spread across London, Tokyo and New York) 2) Low figures - 300-500 hits/s at peak. A lot of our backends have very high latency though (~200ms just to connect to the server), and are predominantly static content, so caching speeds up application loading considerably. 3) Global corporate intranet (Finance sector) 4) No, but we have looked at it. The lack of user/entitlements awareness put us off it, and we moved the logic to another layer. 5) #1 - GZIP compression #2 - SSL support (I know, I know...) - both for accepting incoming connections, and for outgoing connections to backends Thanks, Sam On 29 January 2010 14:48, Per Andreas Buer wrote: > Hi list. > > I'm working for Redpill Linpro, you might have heard of us - we're the main > sponsor of Varnish development. We're a bit curious about how Varnish is > used, what features are used and what is missing. What does a typical > installation look like? The information you would choose to reveal to me > would be aggregated and deleted and I promise you I won't use it for any > sales activities or harass you in any way. We will pubish the result on this > list if the feedback is significant. If you have the time and would like to > help us please take some time and answer the questions in a direct mail to > me. Thanks. > > 1) How many servers do you have running Varnish? > > 2) What sort of total load are you having? Mbit/s or hits per second are > preferred metrics. > > 3) What sort of site is it? > *) Online media > *) Cooperate website (ibm.com or similar) > *) Retail > *) Educational > *) Social website > > 4) Do you use ESI? > > 5) What features are you missing from Varnish. Max three features, > prioritized. Please refer to > http://varnish-cache.org/wiki/PostTwoShoppingList for features. > > > > -- > Per Andreas Buer > Redpill Linpro Group - Changing the Game > > Mobile +47 958 39 117 / Phone: +47 21 54 41 21 > ___ > varnish-misc mailing list > varnish-misc@projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
1)8 2)600-800Mbit/s 60Kreq/s ( there is still big margin on that 8 servers) 3)sport realtime results 4)no 5)gracefull restart( not in meaning of persistent cache, but in the meaning of not flushing all clients and letting them wait for tens of seconds till enough threads appear.) Per Andreas Buer napsal(a): > Hi list. > > I'm working for Redpill Linpro, you might have heard of us - we're the main > sponsor of Varnish development. We're a bit curious about how Varnish is > used, what features are used and what is missing. What does a typical > installation look like? The information you would choose to reveal to me > would be aggregated and deleted and I promise you I won't use it for any > sales activities or harass you in any way. We will pubish the result on this > list if the feedback is significant. If you have the time and would like to > help us please take some time and answer the questions in a direct mail to > me. Thanks. > > 1) How many servers do you have running Varnish? > > 2) What sort of total load are you having? Mbit/s or hits per second are > preferred metrics. > > 3) What sort of site is it? > *) Online media > *) Cooperate website (ibm.com or similar) > *) Retail > *) Educational > *) Social website > > 4) Do you use ESI? > > 5) What features are you missing from Varnish. Max three features, > prioritized. Please refer to > http://varnish-cache.org/wiki/PostTwoShoppingList for features. > > > ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
RE: Survey; how do you use Varnish?
1) How many servers do you have running Varnish? 8 servers (2 sites x 4 servers), load balanced behind F5 GTM. We aim to be able to lose a site AND suffer a hardware failure and keep on truckin'. We could probably run on one or two servers at a push, but our backend would most likely explode before Varnish broke a sweat. Each server is a quad core Xeon w/ 16G RAM. We have a fairly large working set. (+1 Varnish server for Dev / test, which is a VM) 2) What sort of total load are you having? Mbit/s or hits per second are preferred metrics. ~900 req/sec at peak per Prod server / 60Mbps 3) What sort of site is it? *) Retail <= online auctions 4) Do you use ESI? No. 5) What features are you missing from Varnish. Varnishlog filtering language (or other enhancements in this area) Dynamic stats counters Large dataset performance improvements -Original Message- From: varnish-misc-boun...@projects.linpro.no [mailto:varnish-misc-boun...@projects.linpro.no] On Behalf Of Martin Boer Sent: Tuesday, 2 February 2010 10:23 p.m. To: Per Andreas Buer Cc: varnish-misc@projects.linpro.no Subject: Re: Survey; how do you use Varnish? 1) One active server. We have another one as hot standby. 2) 50Mbit, 200 requests/second max. Most of the time it's 10Mbit, 40 requests/second which isn't much. 3) Internet touroperator. 4) Nope 5) Automatic refreshing of data without having the endusers have to wait for the response. The reason we use varnish most is because our website has complex, timeconsuming queries to backend systems. The answers to these queries do vary several times per day but are still cachable. Of course varnish also helps te bring down the load on those backend systems but the main use is that varnish gives the endusers a lightning fast prerendered interactive experience which is a paradox. We like working paradoxes. Something like 'refresh pages after object.prefetch seconds if at least someone requested that object the last object.ttl seconds' where object.ttl is larger then object.refresh. So an object might be prefetched even a couple of times without anyone being interested but will be removed from the cache eventually after object.ttl has expired. Regards, Martin Boer Per Andreas Buer wrote: > Hi list. > > I'm working for Redpill Linpro, you might have heard of us - we're the main > sponsor of Varnish development. We're a bit curious about how Varnish is > used, what features are used and what is missing. What does a typical > installation look like? The information you would choose to reveal to me > would be aggregated and deleted and I promise you I won't use it for any > sales activities or harass you in any way. We will pubish the result on this > list if the feedback is significant. If you have the time and would like to > help us please take some time and answer the questions in a direct mail to > me. Thanks. > > 1) How many servers do you have running Varnish? > > 2) What sort of total load are you having? Mbit/s or hits per second are > preferred metrics. > > 3) What sort of site is it? > *) Online media > *) Cooperate website (ibm.com or similar) > *) Retail > *) Educational > *) Social website > > 4) Do you use ESI? > > 5) What features are you missing from Varnish. Max three features, > prioritized. Please refer to > http://varnish-cache.org/wiki/PostTwoShoppingList for features. > > > > ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
]] Václav Bílek | 5)gracefull restart( not in meaning of persistent cache, but in the | meaning of not flushing all clients and letting them wait for tens of | seconds till enough threads appear.) You can tune the thread_pool_add_delay parameter to at least bring the time down somewhat. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
Tollef Fog Heen napsal(a): > ]] Václav Bílek > > | 5)gracefull restart( not in meaning of persistent cache, but in the > | meaning of not flushing all clients and letting them wait for tens of > | seconds till enough threads appear.) > > You can tune the thread_pool_add_delay parameter to at least bring the > time down somewhat. > alredy set to "1" bud for 4000 threads it takes around 20seconds till fully working state ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
Hi. A short summary of what I've found. This was a rather superficial survey - I hope we can come back with a more thorough one after VUG2. A typical varnish installation is either 2-4 dedicated servers or 10-20 servers running more or less the complete stack on every server. Only 2 out of 14 use ESI but three are planning to deploy ESI soon. I personally think ESI adoption is relatively high considering ESI is more or less a Varnish-only feature (few Akamai customers seem to use ESI because Akamai charges quite a lot extra for enabling it). The most wanted feature is persistence and gzip, 9 points each. The hash director is also much asked about - 6 points. The DNS director is also (to me) surprisingly popular - I believe Kristian has an implementation of it for the 2.0 series so it shouldn't be to long a wait for this one. Better cookie handling is also a common request. Varnish is used everywhere. No area of business stood out - maybe except "online media" which is quite an empty term since it covers a lot. -- Per Andreas Buer Redpill Linpro Group - Changing the Game Mobile +47 958 39 117 / Phone: +47 21 54 41 21 ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
Sorry for the late reply. 1) About 200 servers 2) Averages 2500 hits/sec over the day, approx 35% hit ratio (which is awesome for web hosting) 3) Web hosting 4) No 5) #1 Allow for 304 revalidation of cache content. #2 There was an earlier reply that mentioned automatic revalidation for warm content, i.e. after the TTL expires, leave content in cache if it hasn't changed till some amount of time has passed. Obviously implies #1 #3 Similarly to #2, if unallocated memory is under a certain threshold, allow stale content to live indefinitely, which also implies #1 ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
1) How many servers do you have running Varnish? 30 2) What sort of total load are you having? Mbit/s or hits per second are preferred metrics. 2gbt per second 3) What sort of site is it? ecommerce *) Online media *) Cooperate website (ibm.com or similar) *) Retail *) Educational *) Social website 4) Do you use ESI? no 5) What features are you missing from Varnish. Max three features, prioritized. Please refer to http://varnish-cache.org/wiki/PostTwoShoppingList for features. On Fri, Jan 29, 2010 at 11:48 AM, Per Andreas Buer < per.b...@redpill-linpro.com> wrote: > Hi list. > > I'm working for Redpill Linpro, you might have heard of us - we're the main > sponsor of Varnish development. We're a bit curious about how Varnish is > used, what features are used and what is missing. What does a typical > installation look like? The information you would choose to reveal to me > would be aggregated and deleted and I promise you I won't use it for any > sales activities or harass you in any way. We will pubish the result on this > list if the feedback is significant. If you have the time and would like to > help us please take some time and answer the questions in a direct mail to > me. Thanks. > > 1) How many servers do you have running Varnish? > > 2) What sort of total load are you having? Mbit/s or hits per second are > preferred metrics. > > 3) What sort of site is it? > *) Online media > *) Cooperate website (ibm.com or similar) > *) Retail > *) Educational > *) Social website > > 4) Do you use ESI? > > 5) What features are you missing from Varnish. Max three features, > prioritized. Please refer to > http://varnish-cache.org/wiki/PostTwoShoppingList for features. > > > > -- > Per Andreas Buer > Redpill Linpro Group - Changing the Game > > Mobile +47 958 39 117 / Phone: +47 21 54 41 21 > ___ > varnish-misc mailing list > varnish-misc@projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
