Hi Jim, We appreciate and have gotten the message about i/o bounding (and network bounding as well) versus space. Jacek has spent the last several weeks working on a disk i/o model that presumes large RAM for caching indices and sufficient CPU to take advantage (document-1990). It is also based on a model that Tim Axelrodd, Kem Cook, Zeljko Ivezic, Chris Smith, and Kirk Borne have developed for source detections, object counts, and a data schema that optimizes spatial queries and temporal queries.
We have also examined the required network bandwidth to serve that data out to users and convinced ourselves with the current cost trends that we can afford both the disks and network bandwidth. Now we are trying to see what CPU we need to drive that number of disks/queries with the RAM-based indices, as we have to cost that for our proposal. Your calculations below are very relevant to that question. We have been working from a model of: CPU load in TFLOPS, empirically derived from query performance tests and CPU FLOPS/$ trends derived from industry input Don Dossa from LLNL is actually our lead person on the industry trends and CPU analysis, so I will ensure that he sees this and uses your input. Thanks! Jeff > From: "Jim Gray" <[EMAIL PROTECTED]> > Reply-To: LSST Data Management <[email protected]> > Date: Tue, 25 Jul 2006 19:49:03 -0700 > To: "Ray Plante" <[EMAIL PROTECTED]>, "LSST Data Management" > <[email protected]> > Subject: RE: [LSST-data] calcuating flops [TechAssessWG] > > For simple predicates, typical of SDSS, involving at most a few > transcendental functions and lots of math on about 10 fields we can > search at 1 M records/second/ GHz (that's about 1,000 > instructions/record with a clocks/instruction (cpi) > of about 1 (typical of decision support system (dss) workloads). > (really simple predicates go 3x faster). > GHz/chip is going to increase fast (due to multicore -- probably to 64 > cores by 2014 (the new event horizon). > And this search stuff is embarrassingly parallel. > So we get about 128 GHz/chip in that time frame if you can use 64 > threads and so 128 M records/second/chip. > If your records are 100 bytes (indices and vertical partitioning and > compression are your friends). > Then you have a 12GB/s/chip search engine with a one-chip cpu. > That is about 60 disks / cpu chip (and a LOT of RAM chips and bandwidth > per cpu). > > A good book about IPS (instructions per second) = CPI * clocks/second > is > Patterson-Hennessey: > http://www.amazon.com/gp/product/1558603298/103-4189199-8747028?v=glance > &n=283155 > > But... 60 disks per cpu chip should give you a sense that you are IO > BOUND not CPU bound. > And ... Disks dominate the cost of your system. > Indeed, most internet search guys "waste" cpus by having a mere 8 or 16 > disks per cpu chip. > > So... To repeat.... Its the IO and memory bandwidth that is going to be > your problem. > > NOT cpus, NOT disk space. > > > Jim Gray > Microsoft Research, Suite 1690, 455 Market, SF CA 94105, tel: 415 778 > 8222 fax: 425 706 7329 [EMAIL PROTECTED] > http://research.Microsoft.com/~gray > > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Ray Plante > Sent: Tuesday, July 25, 2006 9:49 AM > To: LSST Data Management > Subject: [LSST-data] calcuating flops [TechAssessWG] > > Keywords: TechAssessWG > > Hi folks, > > Here's how I estimated the flops needed for Jacek's CPU requirement > estimate for supporting DB searching. This was based on lore gleaned > from a Google search on the subject. I understand that the > theoretical/peak flops of a cpu is usually 2 times the nominal clock > rate, so... > > > Psus = 2 * Scpu * Nproc * Rsus > > where Psus = the sustained performance in flops > Scpu = the speed of the CPU used (1.8 GHz) > Nproc = the number of processors needed (5329) > Rsus = reality factor Psus/Ppeak; which I picked as 65% from > an > unscientific sampling of PC clusters. > > As I mentioned, I'm no expert on this, so your corrections are welcome. > > > cheers, > Ray > > > _______________________________________________ > LSST-data mailing list > [email protected] > http://www.lsstmail.org/mailman/listinfo/lsst-data > > _______________________________________________ > LSST-data mailing list > [email protected] > http://www.lsstmail.org/mailman/listinfo/lsst-data > _______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
