The bottleneck you would typically see is interrupts from network traffic (especially if you're tracking connections), bandwidth limits, slow backends, too many keepalive sessions, and pthread stack size. Some of those can exacerbate the thread count and memory usage on an already stodgy pthreads library. And these limitations are pretty much endemic of any proxy or server on every platform, IMHO.
Perhaps a combination of thread- and event-based workers could scale more smoothly? In any case, we've seen Varnish saturating GigE with about 2-3 cores (out of 8). From real life experience in our usage case, Varnish would saturate 10GigE on an 8-core box. Few people need (or would want) to saturate 1GigE on a single box, much less 10GigE. I guess my point is that certain use cases (some valid, some not, some involving bad pthread libraries in distributions (lots of them out there!)) could cause specific scalability issues, and those specific cases should be the focus. "Varnish only scales to two cores" is a generality that my experience refutes, for what it's worth. Ken On Sep 9, 2009, at 9:30 AM, Poul-Henning Kamp wrote: > In message <[email protected]>, =?UTF-8?B? > VsOhY2xhdiBCw61sZWs=?= writes: >> Helo >> >> >> what are your experiences using varnish on multi CPU systems on >> Linux/Freebsd? >> >> my experience in Linux on 8core is that varnish never gets more than >> 20% of all CPUs, only vhen it is overloaded, then it takes all CPU >> ( but >> berformance drops). > > That is pretty typical behaviour. > > In general Varnish does not need much CPU, it needs only an average > of seven sytem calls for a cache hit (last I looked). > > The problem is that when things pile up, for whatever reason, Varnish > sort of climbs the pile. > > Various ideas have been floated, how to deal better with that, but > no really good idea has been found yet. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > [email protected] | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by > incompetence. > _______________________________________________ > varnish-misc mailing list > [email protected] > http://projects.linpro.no/mailman/listinfo/varnish-misc _______________________________________________ varnish-misc mailing list [email protected] http://projects.linpro.no/mailman/listinfo/varnish-misc
