On Friday, August 9, 2002, at 05:22 AM, Jerry Asher wrote: > Back in '99, Philip Greenspun wrote: > >> America Online is fielding 28,000 hits per second across all of its >> various Web services and servers > > I am curious if anyone knows what the configuration looked like at that > time. How many servers was AOL using? What were they?
PhilG's comment was somewhat misleading; there was no single server nor single cluster taking 28,000 hits per second. This was the aggregation of traffic through www.aol.com, www.digitalcity.com, www.primehost.com, and others, including the web proxies AOL operated for people using the AOL client to surf the web. Each of these was a separate installation running on different stuff. At one time, I think www.aol.com was running on a bunch of HP-UX boxes (I don't know how many), but then maybe it moved to IRIX boxes, and now it's on something else -- maybe Solaris, although there might be some HP-UX back in the mix. > On a somewhat related topic, I would like to know how many persistent > connections a cough, typical linux server, cough, might be expected to > reliably service, and what are the limitations involved. In one of the ArsDigita papers, someone said the optimal number for MaxThreads was 10, but didn't really specify how many processors were used in the configuration. The limits for traffic have to do with lock contention, which I've never really seen measurements for (other than the telemetry page, which basically says "if you have a lot of waiting for this lock, you need to do something else), which drives how high you can set MaxThreads, and then the page service time. If you can service a page in 20 ms, and you don't suffer from contention at 10 threads, then you can probably realistically do 500 connections per second. On a 6-way Solaris box, I've run MaxThreads at 30 with no apparent contention, but the page service times dominated the service to such an extent that service was measured in seconds per page, not pages per second. Of the ACS-derived sites I've seen, this seems to be a consistent pattern. Jim has recently said on this list that one of the reasons for moving to poll is that they have boxes which are servicing more than 1024 concurrent connections (select, on a default kernel, usually can't handle an fd_set bigger than 1024, and I'm guessing that using custom kenels is not workable for what Jim is doing), so we know it goes that far. That's probably Sun hardware. On a uniprocessor Linux box, running a 2.4 kernel (which has much spiffier networking than the 2.2 kernels) and probably not running ACS, you can give apparent concurrent service levels in the thousands, although I still think you have to set MaxThreads lower than you would think is reasonable (somewhere between 20 and 50). Pete.
