On Tue, Feb 17, 2009 at 4:57 PM, Sean Gorman <[email protected]> wrote: > On a related note has anyone done mathematical modeling for extrapolating a > supportable user base from a concurrent user threshold. > > For example a server with an application will support 100 concurrent users > and under normal usage patterns would support a universe of 10,000 users with > a 95% certainty that 100 of these users would not all click for the same > action at the same time exactly. >
To speak more specifically - we build 'user models' that estimate application usage extracted from server logs. We build personas such as data producer, data consumer, map producer, map consumer, and others. You can estimate or specify things like acceptable wait times, user efficiency, and deviation from this pattern. We then do variations across these various personas to get ideas of limits on server and software capacity. Then, based on the customer/use, we can estimate the composition of Producers/Consumers of the various components and spec a system. Then using traditional engineering practices, we multiply that by 2 (or other 'fudge factor') Obviously the general advice so far is how to continue scaling out, or deal with unintended spikes (and planning for those unintended spikes). How to prepare for hardware failure, scaling beyond a single computer, and more. But it's important initially to scope out the usage patterns and then comparing those against the options. This would make for a great paper/presentation at FOSS4G - and get yourself a trip to Sydney. :) Andrew > ----- Original Message ----- > From: "Jon Gallagher" <[email protected]> > To: "Eric Wolf" <[email protected]> > Cc: [email protected], [email protected] > Sent: Tuesday, February 17, 2009 3:51:03 PM GMT -05:00 US/Canada Eastern > Subject: Re: [Geowanking] High-volume Server experience > > You're right to focus on the "soft" side, as a bad software architecture can > keep you from exploiting new and better architecture. For example, setting > up .Net and IIS incorrectly and making the wrong calls forces you to use > local storage for code and data rather than using network addressable > storage (NAS). Tkaing the wrong approach on session management hoses your > ability to do true load-balancing. > > Make sure that you break out your application from our data access, and > design your session management so you can pass users between servers and > you'll be fine. > > Jon > > On Tue, Feb 17, 2009 at 12:42 PM, Eric Wolf <[email protected]> wrote: > >> Fortunately, the company I work for has a pretty good budget for >> hardware. We just spent $787B on infrastructure improvements. >> >> Seriously though, I'm doing research for future versions of USGS >> spatial data systems. The problems are always grounded more in what >> can be done in software. The hardware can generally be trended via >> Moore's Law. >> >> I'm trying to map out a fuzzy relationship between usage and >> capabilties, dealing with some rather largish datasets. >> >> -Eric >> >> -=--=---=----=----=---=--=-=--=---=----=---=--=-=- >> Eric B. Wolf 720-209-6818 >> USGS Geographer >> Center of Excellence in GIScience >> PhD Student >> CU-Boulder - Geography >> >> >> >> >> On Tue, Feb 17, 2009 at 1:28 PM, Mano Marks <[email protected]> wrote: >> > Obviously software is important, but I think far more important is the >> > hardware. Software can be configured to load balance between more than >> > one machine if necessary. >> > >> > On Tue, Feb 17, 2009 at 12:02 PM, P Kishor <[email protected]> wrote: >> >> On Tue, Feb 17, 2009 at 1:50 PM, Eric Wolf <[email protected]> wrote: >> >>> Thanks Chris, >> >>> >> >>> I'm trying to dig up some better stats to put out there for >> >>> comparison. And maybe a simple "hits/month" is meaningless in this >> >>> context. >> >>> >> >>> And maybe a question I should really try to answer is "What level of >> >>> traffic would be considered >> >>> 'successful' for a national geospatial basemap?" >> >>> >> >> >> >> >> >> Which, in turn, begs the question, "What do you mean by 'successful'?" >> >> A better way to go about this would be to estimate what kind of >> >> traffic you estimate, and what would the majority of this traffic be >> >> doing. >> >> >> >> Most modern technology (hardware with proper disks, fast network >> >> cards, etc.), correctly configured, would happily handle most >> >> reasonable (non-Yahoo/Google/Flickr/Twitter) traffic. MapServer, for >> >> example, has stood many an onslaught. The same would likely be true of >> >> GeoServer and the ilk. >> >> >> >> >> >>> >> >>> >> >>> >> >>> >> >>> On Tue, Feb 17, 2009 at 12:45 PM, Christopher Schmidt >> >>> <[email protected]> wrote: >> >>>> On Tue, Feb 17, 2009 at 12:23:37PM -0700, Eric Wolf wrote: >> >>>>> Hey All, >> >>>>> >> >>>>> I'm trying to gather some data on how well the various map servers >> >>>>> (GeoServer, MMS, ArcIMS, MapGuide, GeoMedia WebMap, etc) handle large >> >>>>> volumes of users - on the order of 1M hits/month. >> >>>> >> >>>> ... a *month*? Doing what, I guess is the question. I serve 2 million >> >>>> tiles a day off one old server; based on some back of the envelope >> >>>> numbers, I'd say that's probably about 50,000 users a day. (Possibly >> >>>> more; I don't have stats handy.) In the big scheme of things, this is >> >>>> practically nothing: OpenStreetMap, for example, serves 20 times this >> >>>> many tiles (ranging up towards 400 requests/second). >> >>>> >> >>>> http://munin.openstreetmap.org/openstreetmap/tile.openstreetmap.html >> >>>> >> >>>> Clearly, doing interactive editing and the like are much different, >> but >> >>>> even OSM's editing stats far outstrip '1M hits/month': In the past >> >>>> month, 7500 users have made something along the lines of 15-20 million >> >>>> edits, with some days reaching more than 2 million edits in a single >> >>>> day: >> >>>> >> >>>> http://www.openstreetmap.org/stats/data_stats.html >> >>>> http://wiki.openstreetmap.org/index.php/Statistics >> >>>> >> >>>> Regards, >> >>>> -- >> >>>> Christopher Schmidt >> >>>> MetaCarta >> >>>> >> >>> >> >> >> >> >> >> -- >> >> Puneet Kishor http://www.punkish.org/ >> >> Nelson Institute for Environmental Studies http://www.nelson.wisc.edu/ >> >> Open Source Geospatial Foundation (OSGeo) http://www.osgeo.org/ >> >> >> >> _______________________________________________ >> >> Geowanking mailing list >> >> [email protected] >> >> http://geowanking.org/mailman/listinfo/geowanking_geowanking.org >> >> >> > >> > >> > >> > -- >> > Mano Marks >> > Geo Developer Advocate >> > Google, Inc. >> > [email protected] >> > http://twitter.com/ManoMarks >> > >> > _______________________________________________ >> > Geowanking mailing list >> > [email protected] >> > http://geowanking.org/mailman/listinfo/geowanking_geowanking.org >> > >> >> _______________________________________________ >> Geowanking mailing list >> [email protected] >> http://geowanking.org/mailman/listinfo/geowanking_geowanking.org >> > > > > -- > Jon Gallagher > cell: 619-318-5999 > Skype: jon.gallagher > Twitter: JonGal > > _______________________________________________ > Geowanking mailing list > [email protected] > http://geowanking.org/mailman/listinfo/geowanking_geowanking.org > > _______________________________________________ > Geowanking mailing list > [email protected] > http://geowanking.org/mailman/listinfo/geowanking_geowanking.org > -- Andrew Turner mobile: 248.982.3609 [email protected] http://highearthorbit.com http://geocommons.com Helping build the Geospatial Web Introduction to Neogeography - http://oreilly.com/catalog/neogeography _______________________________________________ Geowanking mailing list [email protected] http://geowanking.org/mailman/listinfo/geowanking_geowanking.org
