Re: Varnish and mp3 streaming on demand
Ole Laursen writes: > Andreas Fassl writes: > > Hi, especially the caching is very important for us, because we want to > > keep traffic away from the mp3 repository server. > > So you recommend: > > Client requests streaming on demand mp3 > > - lighthttpd does streaming and requests from > > - varnish as reverse proxy/cache from > > - mp3 repository I reread your original post, and I can see know that you're probably referring to "Range header support" on http://varnish.projects.linpro.no/wiki/PostTwoShoppingList which is presumably missing. So forget the last thing I said. Ole ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
"Poul-Henning Kamp" writes: > No, we never stream pass, the current design is aimed at freeing > up the backend as fast as possible, and we only start to transmit > the object to the client once we have all of it. Perhaps it is time to consider adding a third option: "stream", which streams the current request, then reverts to normal operation. DES -- Dag-Erling Smørgrav - d...@des.no ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
Poul-Henning Kamp writes: > >Right now pass in vcl_recv is streaming, right? > >You're only talking about if the > >object is entered into the cache? Or both cases? > > No, we never stream pass, the current design is aimed at freeing > up the backend as fast as possible, and we only start to transmit > the object to the client once we have all of it. OK, thanks! That's good to know. Also that you're fixing it. :) Ole ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
In message , Ole Laursen writes: >Poul-Henning Kamp writes: > >> This is necessary to be able to decide, per object, if it should be >> stored in temporary (malloc) or persistent (disk) storage. >> >> With some extra work, this will allow pass to become streaming. > >Right now pass in vcl_recv is streaming, right? >You're only talking about if the >object is entered into the cache? Or both cases? No, we never stream pass, the current design is aimed at freeing up the backend as fast as possible, and we only start to transmit the object to the client once we have all of it. This is the same for both caching and passing. pipe is, by nature of how it works, streaming. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
Poul-Henning Kamp writes: > This is necessary to be able to decide, per object, if it should be > stored in temporary (malloc) or persistent (disk) storage. > > With some extra work, this will allow pass to become streaming. Right now pass in vcl_recv is streaming, right? You're only talking about if the object is entered into the cache? Or both cases? Ole ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
Andreas Fassl writes: > Hi, especially the caching is very important for us, because we want to > keep traffic away from the mp3 repository server. > So you recommend: > Client requests streaming on demand mp3 > - lighthttpd does streaming and requests from > - varnish as reverse proxy/cache from > - mp3 repository Well, I'd setup lighttpd on the MP3 repository server to serve the MP3 files. So client -> varnish cache server -> lighttpd on mp3 repository server However, why do you want to keep traffic away from the MP3 repository? If the server can handle it, letting clients talk directly to it is clearly the most efficient way of serving the files. If you stream MP3, it's probably < 256 kbit. If you can get 1 Gbit out of your repository server, we're talking maybe 3000-4000 simultaneous listeners at 256 kbit. That sounds like a lot. I guess it makes sense if you're a really large radio station or music site and you need to scale up beyond that single server. You could then setup several Varnish servers and have them cache the files automatically instead of distributing the files manually over the nodes. Hmmm. Ole ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
In message <49945b51.90...@progis.de>, Andreas Fassl writes:
>Hi, especially the caching is very important for us, because we want to
>keep traffic away from the mp3 repository server.
>So you recommend:
>Client requests streaming on demand mp3
>- lighthttpd does streaming and requests from
>- varnish as reverse proxy/cache from
>- mp3 repository
For your information:
One of the changes I'm working on right now is to move vcl_fetch{}
up to after receiving the headers from the backend, but before
dealing with the backend.
This is necessary to be able to decide, per object, if it should be
stored in temporary (malloc) or persistent (disk) storage.
With some extra work, this will allow pass to become streaming.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
Hi, especially the caching is very important for us, because we want to
keep traffic away from the mp3 repository server.
So you recommend:
Client requests streaming on demand mp3
- lighthttpd does streaming and requests from
- varnish as reverse proxy/cache from
- mp3 repository
Best regards
Andreas
Ole Laursen schrieb:
Andreas Fassl writes:
after reading the docs it looks like I need an apache server to serve
the cached mp3 content for streaming on demand.
Any experience in configuration of this setup?
No, but I have set up a site with videos streamed with a Flash widget. We let
the video files pass through the cache unaltered with something like
if (req.url ~ "^/media/content") {
pass;
}
and send the request to a lighttpd backend. Lighttpd is easy to setup to serve
static files, modest in resource usage for this task and scales very well. The
disk cache in the operating system makes sure the files are in memory - it's
just a matter of having enough RAM to keep the requested data in memory. For
this reason, I don't think there's any point in using Varnish's cache in this
kind of setup.
Ole
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
Ole Laursen writes: > just a matter of having enough RAM to keep the requested data in memory. For > this reason, I don't think there's any point in using Varnish's cache in this > kind of setup. In other words, unless you need some other feature of Varnish, you might as well route the requests directly to lighttpd. In my case, we had one server only so Varnish was convenient for multiplexing port 80 in addition to caching the dynamic HTML parts from an Apache web app. Ole ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish and mp3 streaming on demand
Andreas Fassl writes:
> after reading the docs it looks like I need an apache server to serve
> the cached mp3 content for streaming on demand.
> Any experience in configuration of this setup?
No, but I have set up a site with videos streamed with a Flash widget. We let
the video files pass through the cache unaltered with something like
if (req.url ~ "^/media/content") {
pass;
}
and send the request to a lighttpd backend. Lighttpd is easy to setup to serve
static files, modest in resource usage for this task and scales very well. The
disk cache in the operating system makes sure the files are in memory - it's
just a matter of having enough RAM to keep the requested data in memory. For
this reason, I don't think there's any point in using Varnish's cache in this
kind of setup.
Ole
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
