The usage of varnish, revisited
Dump from my blog posting at http://ingvar.blog.linpro.no/ Ingvar The usage of varnish, revisited This is more or less a repost, with updated numbers. Some months have passed, and it is time to run my poking scripts again, looking for sites that run Varnish. There is no deep magic here. I just parse the available top lists that I know of, and peek at the HTML headers of the sites that are listed. If there are subsites linked from the front page of the site, I scan them too. This means that twitter.com shows up, though Twitter only runs Varnish on its search site. Subsites with a Varnish match are shown in parenthesis in the results. For the Nordic countries, I have found quite good lists, that is, upload result lists from the probably most visited media sites in the respective countries. Remember of course, that these are generally pay-to-be-included lists, and there may exist sites with far more hits than the ones listed. For a global overview, I have used Alexa. Now for the results. Varnish is sponsored by large Norwegian sites, so it is no big surprise that there are a lot of hits in Norway. Of the TNS Gallup top list, Varnish runs at 36 of the top 100 sites. That’s 3 up since my last probe. For Denmark, I use FDIM’s list. From May, we are up from 3 to 11 sites in the top 100. For Finland, I use TNS’ numbers again. No changes there. For Sweden, I use the KIA Index list. I now probe the 200 top sites, so there are several more varnish sites on the list. In the top 100, we are up from 8 to 9. For the Alexa’s World top 500 list, I have tweaked my filters a bit, and the list is up from 7 to 17 sites since my last probe in May. World Domination, here we come! The whole World: Alexa's global top 500 list Place 12 Varnish running on twitter.com (http://integratedsearch.twitter.com/search.html) Place 47 Varnish running on photobucket.com (http://blog.photobucket.com/) Place 90 Varnish running on mixi.jp (http://img.mixi.jp/static/css/basic/logout_quirks) (and others) Place 101 Varnish running on weather.com (http://www.weather.com/) (weather.com) Place 107 Varnish running on globo.com (globo.com) (http://www.globo.com/) Place 111 Varnish running on ifeng.com (http://big5.ifeng.com/gate/big5/www) Place 138 Varnish running on answers.com (http://www.answers.com/) (answers.com) Place 170 Varnish running on orange.fr (http://r.orange.fr/r/Eorangepublicite) Place 179 Varnish running on hulu.com (hulu.com) Place 199 Varnish running on wikia.com (wikia.com) Place 213 Varnish running on xinhuanet.com (http://big5.xinhuanet.com/gate/big5/www) Place 290 Varnish running on people.com.cn (http://bbs1.people.com.cn/boardList) (and others) Place 344 Varnish running on tuenti.com (http://estaticos1.tuenti.com/layout/web2/images/favicon) Place 428 Varnish running on chinanews.com.cn (http://www.chinanews.com.cn/test/index_back_d.html) (and others) Place 433 Varnish running on mercadolivre.com.br (http://veiculos.mercadolivre.com.br/) (and others) Place 447 Varnish running on mercadolibre.com.mx (http://tendencias.mercadolibre.com.mx/) (and others) Place 456 Varnish running on break.com (break.com) In my last probe, I poked sites all over Europe. With a few exceptions, that was a bit less interesting. Global .com and .net sites tend to cover most of the top 100 entries, as I had only toolbar lists, and it’s not that spectacular that for example people in Serbia and Monte Negro are browsing Twitter, like the rest of the World does. So I have skipped other countries. If you know of any good top list for you country that is not toolbar based, please let me know. All the gory details are available here: http://users.linpro.no/ingvar/varnish/stats-2010-01-18.txt Other more or less worth mentioned sites that is reported to use Varnish but does not show up in my lists, may be Slashdot, WAT TV, The Pirate Bay, JDownloader, e.Republik, WOWwiki, Globo.com, PCWelt.de, BlackPlanet, funnyordie.com and n-tv.de, to name a few. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
RPM-packages of varnish-2.0.5 for RHEL and Fedora available
I have submitted varnish-2.0.5 for Fedora and Fedora EPEL, and updates to the stable releases will be requested, so they will trickle down to the stable repos in a few weeks. For RHEL, both el4 and el5 packages are now in the EPEL testing repo. For those who are too impatient to wait for stable, or want to participate in testing, you can download the package with yum: rhel5# yum --enablerepo=epel-testing update varnish ... or download the package from RedHat: http://download.fedora.redhat.com/pub/epel/testing/ Fedora packages are still pending for testing, but will be visible in a few days, I guess. If you need packages for Fedora now, try http://kojipkgs.fedoraproject.org/packages/varnish/ Bugs in the package can be reported in Red Hat's Bugzilla: http://bugzilla.redhat.com/ or to varnish-d...@projects.linpro.no. Ingvar ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: varnish-misc Digest, Vol 23, Issue 25
* Fredrik Nygren Alright. Wonder then if somebody has any rpm's to share for Redhat EL4 i386 with the patch included? Post a request with a short recap of the discussion to the varnish-dist list, and I'll consider it. Ingvar -- Buddha wears an iPod ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Setting default_ttl in vcl file
I take this off varnish-misc as it is quite distro specific. * Ingvar Hagelund The RedHat and Debian init scripts reads default values from their respective defaults file, that is, for RedHat, /etc/sysconfig/varnish, and for Debian, /etc/defaults/varnish. * Dag-Erling Smørgrav The problem is that if you define DAEMON_OPTS, none of the other knobs work, right? FreeBSD at least has that issue: if you define varnishd_flags, you have to specify everything manually, since the other knobs only serve to populate the default value for varnishd_flags. At least for RedHat, there are no other knobs. All the startup configuration is in the file. There are no hard coded values in the init script. The init script won't start varnishd if DAEMON_OPTS is empty, or the config file does not exist. if [ $DAEMON_OPTS = ]; then echo \$DAEMON_OPTS empty. echo -n Please put configuration options in /etc/sysconfig/varnish echo_failure The configuration file is fairly documented with the most used options, and points to the manpage for more advanced users. I would not know if this is suitable for FreeBSD ports, but I think it's quite a clean method to ensure that the user knows what she wants to change before changing it. As for myself, I dislike magic invisible default values. Ingvar -- Buddha wears an iPod ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Setting default_ttl in vcl file
* Steve Scally Now my question, is there a more direct way to change the default_ttl either through configuration files or .vcl file? If these are pretty much the only three ways that is fine, I just wanted to make sure I wasn't skipping over any options. Thank you in advance for your help. * Dag-Erling Smørgrav The default TTL can be configured on the command line with -t ttl or (as any other run-time parameter) with -p default_ttl=ttl. The FreeBSD rc script does not currently have a variable for setting the default TTL directly, nor a mechanism for setting run-time parameters in general. I could either add code to the rc script to read in a list of run-time parameters and pass them on the command line, or we could add code to varnishd to read initial values from a file. I prefer the former, but we need to add similar code to the Debian and RedHat init scripts as well. The RedHat and Debian init scripts reads default values from their respective defaults file, that is, for RedHat, /etc/sysconfig/varnish, and for Debian, /etc/defaults/varnish. In that file, you can add any command line parameter to varnishd, not only those that are commented in the file. Do we need any other mechanism for adding runtime parameters? A full-fledged configuration file parser in varnishd? Ingvar redhat$ tail -4 /etc/sysconfig/varnish ## Alternative 4, Do It Yourself. See varnishd(1) for more information. # # DAEMON_OPTS= ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Setting default_ttl in vcl file
This mail has stayed unanswered on varnish-dist, but I think someone on
varnish-misc may be able to answer.
Ingvar
Steve Scally [EMAIL PROTECTED] said on [EMAIL PROTECTED]
Hello,
I have just started to use varnish and was looking into the default
cache time of an object / item, etc. I know the default_ttl is 120
seconds and I was looking to change that to 3600 seconds. From the
documentation and previous mail archives I have found three ways to
set this. Two ways are through the rc.d and rc.conf files and the
last is through the vcl.
A. /usr/local/etc/rc.d/varnishd file
Create a variable called varnishd_ttl and set it to 3600.
: ${varnishd_ttl=3600}
Edit varnishd_flags and add my new variable
: ${varnishd_flags=-P ${pidfile} -a ${varnishd_listen} -f $
{varnishd_config} -T ${varnishd_telnet} -s ${varnishd_storage} -t $
{varnishd_ttl}}
B. /etc/rc.conf
Create a variable called varnishd_ttl and set it to 3600.
: ${varnishd_ttl=3600}
Edit varnishd_flags and add my new variable
varnishd_enable=YES
varnishd_listen=mb-prv.kcilink.com:80
varnishd_storage=file,/var/tmp,512M
varnishd_config=/usr/local/etc/varnish/morebiz.vcl
varnishd_ttl=3600
varnishd_flags=-P ${pidfile} -a ${varnishd_listen} -f $
{varnishd_config} -T ${varnishd_telnet} -s ${varnishd_storage} -t $
{varnishd_ttl}
c. /usr/local/etc/varnish/default.vcl
*if (obj.ttl 3600s) {
set obj.ttl = 3600s;
}
*not exact just scratch code.
Now my question, is there a more direct way to change the default_ttl
either through configuration files or .vcl file? If these are pretty
much the only three ways that is fine, I just wanted to make sure I
wasn't skipping over any options. Thank you in advance for your help.
Steve
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: docs, howto and best practices
* Sébastien FOUTREL Unfortunately, I already read that articles and it has only a bit of meat to feed my hunger. Where can I found vcl languages docs ? The website and my googling did not gave me good results. I don't think there exists more, at least not in publishable form. Dag-Erling? I agree that documentation of the vcl language, and some best-practice notes and advice is poorly needed for the project. On the other hand, at least, we now have the man pages. We all have to prioritize how we use our time, and that of course includes the main developers, who probably wisely use most of their varnish time on squashing bugs and programming for varnish-2. Dag-Erling, is it possible to push some documentation production up the prio list? I know basics but wants to go further. Varnish is already production used on a 7M hits/D website and it works perfectly. Post your config here and ask about comments for it. Then tell us what else you want to do with varnish, and if you are lucky, some on the list may tell you more. Ingvar -- ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: docs, howto and best practices
* Sébastien FOUTREL I'am relatively new to varnish, but the more I use it, the more I like it. There is only a black point on my pleasure. There is not enough docs available. I browsed the web site and google and did not find enough meat to feed my beast. Where can I find vcl languages docs (apart man pages) ? Could you please give me your ressources on Varnish. I'am ready to use it and, if developers are ok, to feed the wiki to help other people like me. Thank You, Hello, Sébastien Unfortunately, there is still quite a lack of more fleshy documentation than the man pages. John Buswell wrote a nice article in issue #6 of o3magazine. It's available online at http://www.emea.o3magazine.com/pastissues/issue6/ . It includes some starting point vcl code and some general advice. You should start with that article and the man pages, and then be welcome to return to the mailing list for more help. Regards, Ingvar ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Redhat
* jdouglas Are there instructions on installing this on Redhat systems? I am able to get it working on FreeBSD 6.2 but not RHEL 4 with a 2.6 Kernel. When I try to build an rpm from spec i get: (...) RPM build errors: Installed (but unpackaged) file(s) found: /usr/include/varnish/shmlog.h /usr/include/varnish/shmlog_tags.h /usr/include/varnish/stat_field.h /usr/include/varnish/stats.h /usr/include/varnish/varnishapi.h /usr/lib/pkgconfig/varnishapi.pc This is fixed in trunk. The fix will also be included in the upcoming 1.1.1 release. Ingvar -- At barnebokfigurer er snille er bare ljug Mummipappa dater Barbamamma i smug ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Whitescreen issue with varnish 1.0.2-2 (debian etch)
* Kenneth Rørvik I saw the same problems with 1.0.2 on redhat - empty documents returned after a while, possibly related to filling up the disk backend file. * Denis Brækhus 1.0.3 fixed it here, I didn't look deeper into it. 1.0.3 worked here too. I always intended to run 1.0.3, but I wanted to test the etch included package just for the fun of it. It's a bit scary that the package in etch can give a newcomer to Varnish the impression that it is not really stable/production ready though, I am afraid new users could get a bad start. Rumors says 1.0.4 is on the stairs, banging at the door. It will probably include an update to the Debian package. I don't know if changes/updates will trickle into etch over the time. Stig, Lars? Ingvar -- Når alt annet feiler: Symlink ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Packaging: A need for a devel package?
* Matthias Saou From what Dag-Erling answered, it seems like the devel package might make sense. Since the documentation mentioned is missing, I'm going to push the devel package for later/request, as proposed by Poul-Henning. If you do decide to have one, your example above isn't good (at least for Red Hat and Fedora, SuSE, Mandriva and others do things differently), as you would need to have for instance : varnish (the main package with the daemon) varnish-libs varnish-devel (and not varnish-libs-devel) It's kind of strange, as I get different answers every time I ask anybody about this :-) At the moment, I have a package for review for Fedora. Matthias, could you post comments in RedHat Bugzilla, Bug #230275, please? The libs only make sense to split out if some programs could require them without requiring the main daemon. Again, from what Dag-Erling wrote, maybe this would make sense if someone writes a varnishlog file parser. I guess I will to keep the libs package for future use. It's complete, and thus easier to cope with than a non-existing list of header files and documentation. Attached are the files I used to build the latest varnish package I used, in case they can be of any help. Yes, the changes to the initscript and the configuration file are absolutely interesting, though I might insist on putting the config file in /etc/sysconfig. Ingvar -- ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Packaging: A need for a devel package?
I am putting some effort in the (RedHat) rpm package again. I got a question about a devel package. Usually, on RedHat based systems, one typically puts things like static libraries and header files in a devel package, like varnish-libs-devel-1.0.3-5.i386.rpm for instance. Now, I wonder: Would it be appropriate with a devel package at all? Is it thinkable that anyone would use varnish technology to build things outside varnish itself? If so, I could use a list of actual header files and a suggestion on where to put them (/usr/include/varnish?), and maybe some starting point hacking documentation, if that's feasible. If this seems nonsense, please tell, and I'll just skip the devel package. Ingvar -- ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
RPM packages of varnish-1.0.3 available
I have just uploaded rpm packages of varnish-1.0.3 to sourceforge. x86_64, i386 and src flavours available. They should spread to worldwide mirrors in a few hours. I'll see if I get around to build some ppc64 packages too. Most relevant changes in the rpm package: It's now three packages, varnish, libvarnish1 and libvarnish1-devel. This way it conforms more to the RedHat/Fedora standards. There are also some other small changes to make it pass the needle eye of rpmlint. Most intrusive, I converted the manpages to utf8. Dag-Erling, you might want to consider this for the upstream package too. Ingvar -- When everything else fail: Symlink ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
