Hello, As Kem indicated, this will be handled as a design issue. The disk estimates are frozen for now. Thanks!
Jeff > From: "Kem Cook" <[EMAIL PROTECTED]> > Reply-To: LSST Data Management <[email protected]> > Date: Tue, 18 Jul 2006 09:30:47 -0700 (PDT) > To: "LSST Data Management" <[email protected]> > Subject: Re: [LSST-data] updated database storage and disk io estimates > > Keywords: DataAccWG > > The deep stellar detections in the non-plane fields will be in r and i, > and not in z or y due to the decreased QE and fewer number of exposures in > the redder bands. So, there will be stars with r and no y, as well as y > with no r. So, it is not strictly true that the r band detections are a > sub-set of the y. The numbers should be within our margins, but it > should be remembered when designing/building the object catalog. > > cheers, > > Kem > >> >> Hello, >> >> >> I published in docushare the latest version of database >> storage estimates and disk io estimates (Collection-413) >> It includes latest numbers/assumptions provided by Tim, >> and the changes discussed/approved at the database >> telecon (Jul 12). >> >> In summary, we now expect >> - to have 164 TB of data after the first year (down from 600) >> - to need 1.6 K disks for the assumed query load (down from 4 K) >> >> Note that: >> >> 1) there is still a small debate over the galaxy numbers >> for DR20 ("should be more like 18 billion"), and the star >> numbers ("should be the 10 billion from the r-band counts"), >> so further updates are likely. >> >> 2) We are still working on details regarding which table >> should be used by which query (DIASource vs Source). This >> can change disk io (reduce it - now I am using Source) >> >> 3) I introduced small data duplication by extracting 6 rows >> from the Source table into a separate table in order to reduce >> disk io for the near-neighbor query by 85% (which reduced >> overall disk requirement by half). That should be ok given >> we asked for 150% of data size for this sort of >> "persistent overheads" (and are now "using" 8 TB out of 53 TB >> in DR1, or 15%) >> >> >> One worry that I have is that the estimates are now for >> DR1 and in DR1 the Source table is "tiny" (if one can call >> 40-billion-row table "tiny") comparing to what we will >> have to deal with later (for example in release 5 >> it is 10x bigger). The used DR number is selected based on >> number of objects in Object Catalog: we need 10 billion and >> DR1 has over 22 already. >> >> >> Jacek >> _______________________________________________ >> LSST-data mailing list >> [email protected] >> http://www.lsstmail.org/mailman/listinfo/lsst-data >> > > _______________________________________________ > LSST-data mailing list > [email protected] > http://www.lsstmail.org/mailman/listinfo/lsst-data > _______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
