Thanks for all the pointers in this thread... for now I've been pulled to work on something else but I'll be back in the coming weeks.
A few people mentioned Apache, the reason I don't do this with Apache is I want reloading of the configuration to be simple, it seems Varnish reloads are quick and easy. I'll experiment with Varnish inline C next and research some of the other proxies mentioned. Thanks for the pointers.. I'll be back. ;-) Tony On Thu, Aug 5, 2010 at 10:14 PM, Michael Alger <[email protected]> wrote: > On Thu, Aug 05, 2010 at 12:21:07PM -0400, Tony Primerano wrote: > > On Thu, Aug 5, 2010 at 10:50 AM, Per Buer <[email protected] > >wrote: > > > > > On Thu, Aug 5, 2010 at 4:33 PM, Tony Primerano < > [email protected]> > > > wrote: > > > > > > > But what if I have 1000s of backends and I choose them based on > > > > the domain that user's hit varnish with. Is this something > > > > Varnish handles or is it only intended to work with a handful of > > > > backends? > > > > > > It's not built for that. Kristians dns director might help you out > > > a bit, but it just entered trunk (will be in 2.1.4). > > > > > http://kristianlyng.wordpress.com/2010/08/02/varnish-backend-selection-through-dns/ > > > > Looks promising but in the end I think I end up with the same amount of > > code. (a vcl_fetch with 1000s of ifs) > > I don't think there's any avoiding that unless there's some kind of > logical, programmable pattern for mapping the requested host to the > destination host. The examples you gave sound fairly arbitrary, which > just means you'll have to have a great big list. > > How many actual servers (as opposed to domains) are serving as > destinations? If it's still in the thousands, then you're probably > better off looking at a forward-proxy for your solution, although as > mentioned you could put Varnish in front of it to provide caching. > > If the number of servers you connect to is more reasonable, then it > might be plausible to use varnish directly. The Host: header can be > changed by VCL and is independent of the backend definition. > > The main performance issue you'd have, I think, is from the sheer number > of rewrite rules you'd have to have. This is especially the case if > you're having to run a string comparison or regular expression match > over every rule until you find a match. If you control the DNS for the > proxied domain, you can mitigate this a bit by spreading them across > multiple IP addresses. > > For a better solution, I think you'd want to have something that can > look up the requested domain name using a hash table or similar, so you > get essentially constant-time lookups. I don't know of anything that can > do that "out of the box", but you could do it with Varnish using inline > C. It might be worthwhile doing a small-scale test with regular VCL for > the rewrites on a few dozen domains as a proof-of-concept, then if > you're happy with the results look at creating a more scalable rewriter. > > _______________________________________________ > varnish-misc mailing list > [email protected] > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc >
_______________________________________________ varnish-misc mailing list [email protected] http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
