Re: ESI include: src, alt, onerror weirdness
On Feb 13, 2009, at 9:27 AM, Tollef Fog Heen wrote: ]] Florian Gilcher | | How about the unexpected break when retrieving an empty document? That's weird. Can you try again using 2.0.3? Same issue. I'll report it. | [1]: And undocumented. The documentation sadly leads to believe that | include is implemented (as in: full, not half). Do you prefer if I | file a bug or apply for the magic wiki bit and do it on my own? Just tell me your wiki user name and I'll add the bit. Skade Sorry for replying off-list by accident. Regards, Florian Gilcher ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: ESI include: src, alt, onerror weirdness
- Florian Gilcher varnish-li...@andersground.net wrote: Skade Added to the wiki group. -- Redpill Linpro - Changing the Game ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Varnish and virtual hosting
Hi guys I'm fairly new to Varnish and it's goodness and not that brilliant a sysadmin (I'm a developer, really) and I have some questions about how to go about setting up Varnish in a virtual hosting environment. I've had a lot of trouble combining the technologies I use for non-proxied hosting with Varnish and what I'd like to do is start a thread to get other people's experiences then combined all this into a bit of documentation for the wiki. So first, when I host a website in Apache, I use it's log piping capability to parse the output of its access logs for each virtual host through Cronolog, so that we can have one log file per day, organised in folders of hostname/year/month. We then chuck all this data at awstats to generate traffic logs based on this data. When I was trying to set this up to work with varnish, I ran into a couple of problems: 1. There didn't seem to be a way to seperate out log data for each virtual host using varnishncsa. 2. There doesn't seem to be a way to get log data out of varnish in a 'live' fashion or in a historical way; meaning (from my view) that if the server crashes of varnish fails, I have to do some work to get the log data out. 3. There doesn't seem to be a way to process live varnishncsa data via a pipe. If this were possible, then I think I would find this whole process a lot easier. Instead, it seems there are two options: to historically dump log data via an option, or run as a daemon and send output to a file (which cannot be via a pipe). I've read the respective man pages for all of the various applications several times and couldn't find a clear solution, so if anyone on this list has been through something similar, or has any ideas on the subject, please contribute them and I'll write the whole thing up as a kind of how-to. Thanks, Justin -- Redwire Design Limited 54 Maltings Place 169 Tower Bridge Road London SE1 3LJ www.redwiredesign.com [ 020 7403 1444 ] - voice [ 020 7378 8711 ] - fax ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
Andreas Fassl afa...@... writes: Hi, especially the caching is very important for us, because we want to keep traffic away from the mp3 repository server. So you recommend: Client requests streaming on demand mp3 - lighthttpd does streaming and requests from - varnish as reverse proxy/cache from - mp3 repository Well, I'd setup lighttpd on the MP3 repository server to serve the MP3 files. So client - varnish cache server - lighttpd on mp3 repository server However, why do you want to keep traffic away from the MP3 repository? If the server can handle it, letting clients talk directly to it is clearly the most efficient way of serving the files. If you stream MP3, it's probably 256 kbit. If you can get 1 Gbit out of your repository server, we're talking maybe 3000-4000 simultaneous listeners at 256 kbit. That sounds like a lot. I guess it makes sense if you're a really large radio station or music site and you need to scale up beyond that single server. You could then setup several Varnish servers and have them cache the files automatically instead of distributing the files manually over the nodes. Hmmm. Ole ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: running varnish-2.0.2 on Sun
2009/2/12 Tollef Fog Heen tfh...@redpill-linpro.com ]] Rob Ayres | Does anyone have an idea what has caused this? Not really, no. I'm going to get a buildbot slave going on Solaris so we'll hopefully be able to avoid such bugs in the future. If you have found a solution, patches are more than welcome. We set VCC_CC in config.h to be gcc rather than cc and now it compiles the VCL file with no problem. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
Poul-Henning Kamp p...@... writes: This is necessary to be able to decide, per object, if it should be stored in temporary (malloc) or persistent (disk) storage. With some extra work, this will allow pass to become streaming. Right now pass in vcl_recv is streaming, right? You're only talking about if the object is entered into the cache? Or both cases? Ole ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
In message loom.20090213t100923-...@post.gmane.org, Ole Laursen writes: Poul-Henning Kamp p...@... writes: This is necessary to be able to decide, per object, if it should be stored in temporary (malloc) or persistent (disk) storage. With some extra work, this will allow pass to become streaming. Right now pass in vcl_recv is streaming, right? You're only talking about if the object is entered into the cache? Or both cases? No, we never stream pass, the current design is aimed at freeing up the backend as fast as possible, and we only start to transmit the object to the client once we have all of it. This is the same for both caching and passing. pipe is, by nature of how it works, streaming. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
Poul-Henning Kamp p...@... writes: Right now pass in vcl_recv is streaming, right? You're only talking about if the object is entered into the cache? Or both cases? No, we never stream pass, the current design is aimed at freeing up the backend as fast as possible, and we only start to transmit the object to the client once we have all of it. OK, thanks! That's good to know. Also that you're fixing it. :) Ole ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Default behaviour with regards to Cache-Control
Ole Laursen o...@iola.dk writes: I looked up private here http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html and it says Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache. This allows an origin server to state that the specified parts of the response are intended for only one user and are not a valid response for requests by other users Varnish is not a shared cache, it's a surrogate (not covered by RFC2616) DES -- Dag-Erling Smørgrav - d...@des.no ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: [varnish] Re: Default behaviour with regards to Cache-Control
On Feb 13, 2009, at 4:54 AM, Dag-Erling Smørgrav wrote: Ole Laursen o...@iola.dk writes: I looked up private here http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html and it says Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache. This allows an origin server to state that the specified parts of the response are intended for only one user and are not a valid response for requests by other users Varnish is not a shared cache, it's a surrogate (not covered by RFC2616) DES Speaking of which... It would be super handy if Varnish supported Surrogate-Control. I sympathize with the Varnish developers who feel that everything can be done in vcl but some of us really do need the backend to control the surrogate cache behavior on a more granular level. Ric ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
AW: Varnish and mp3 streaming on demand
If I use varnish as cache between mp3 repository and lighttpd, lighttpd will stream the cached contents, that is sufficient for us. If varnish could stream, that would be fine, but I don't think that it is part of the original concept, or? - Originalnachricht - Von: Dag-Erling Smørgrav d...@des.no Gesendet: Fre, 13.2.2009 14:00 An: Poul-Henning Kamp p...@phk.freebsd.dk Cc: Ole Laursen o...@iola.dk ; varnish-misc@projects.linpro.no Betreff: Re: Varnish and mp3 streaming on demand Poul-Henning Kamp p...@phk.freebsd.dk writes: No, we never stream pass, the current design is aimed at freeing up the backend as fast as possible, and we only start to transmit the object to the client once we have all of it. Perhaps it is time to consider adding a third option: stream, which streams the current request, then reverts to normal operation. DES -- Dag-Erling Smørgrav - d...@des.no ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and virtual hosting
3. There doesn't seem to be a way to process live varnishncsa data
via a pipe. If this were possible, then I think I would find this whole
process a lot easier. Instead, it seems there are two options: to
historically dump log data via an option, or run as a daemon and send
output to a file (which cannot be via a pipe).
Wrong. Both varnishlog and varnishncsa will, by default, process live
data and print it to stdout, where you can pipe it to whatever you like.
Agree.
You can post-process varnishncsa data to any post-processing program you want.
Or, for example, using Perl ..you may launch the program and take the
output directly to the files you want.
#!/usr/bin/perl
open (COMMAND, varnishncsa | ) or die Could not open varnishncsa...\n;
while (COMMAND) {
# bla bla bla
print $_; # Print actual line
}
close COMMAND;
Every request is logged with full DN in the URL.
62.151.2.42 - - [29/Jan/2009:16:56:16 +0100] GET
http://www.FULLDOMAIN.info/static HTTP/1.0 503 466 referer
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR
1.1.4322)
So it's not difficult to make a program to separate logs depending on
your needs.
Btw, a program in C made by varnish programmers to accomplish this may
be greatly appreciated by the community ;-)
Kitai
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Configuration question about a domain list for a http.req.host comparaison
Tollef Fog Heen a écrit : ]] Damien Desmarets | I have 1 question for you. I haven't found the response in the | documentation/wiki. I think you have a big job to wrtie a good | documentation ... the current is really poor ! If you want to help out with writing docs or suggest weak points, you are more than welcome. Yes of course, but to begin I need a good documentation to have solids foundations! And after this, be sure i will post modification/add in your wiki documentation. | Is it possible to do this in VCL origin configuration? Not at the moment, no. What I would suggest is you write a small script in your favourite language that generates the VCL for you. Yes sure, and modify the init.d script ... or I can try to add this particularity inside libvcl directly. I'm going to see the difficulty to implement this. Thank for the support, Damien Desmarets ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
