Yes, yes, yes. Though I don't see the clear distinction between Case 2 and Case 3, unless you mean that the SRB metadata are used in Case 3 and the normal LSST DB metadata are used in Case 2.
- Kirk > Date: Fri, 30 Jun 2006 10:04:39 -0700 > From: Arun Jagatheesan <[EMAIL PROTECTED]> > Subject: [LSST-data] File or image retrieval and access patterns? > To: "'LSST Data Management'" <[email protected]> > > This is a question for which I have been trying to find an answer. I would > appreciate any friendly enlightenment about the type of queries astronomers > might have. > > Will the queries by astronomers be purely database-oriented or will it also > lead to retrieval of image files from a data grid type of system? > Case 1: Query the database using only astronomy related attributes to get > result sets only from database. This is purely RDBMS here - traditional SQL > type queries. > Case 2: Query the database using only astronomy related attributes to get > result sets from the database and some files or images. (its more DGMS style > queries here) > Case 3: Query the database using both astronomy related attributes and > storage-related attributes to get result sets both from the database and > image archive > (e.g.) Query on ra, dec etc., and get these metadata, In addition get me the > associated files only iff they can be retrieved with certain QoS or with a > delivery time of these much hours. Or... I don't want 5TB of images coming > out of this query but I am ok with 100GB. > > Thanks in advance, > Arun > ~~~~~~~~~ > Luck is what happens when preparation meets opportunity. > > Arun swaran Jagatheesan > http://www.sdsc.edu/~arun/ > San Diego Supercomputer Center. > (858)822.5452 > > > -----Original Message----- > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] On Behalf Of Jacek Becla > > Sent: Thursday, June 29, 2006 9:24 PM > > To: LSST Data Management > > Subject: [LSST-data] Database telecon Fri Jun 30 > > > > Keywords: DataAccWG > > > > Hi all, > > > > We will have a Database telecon this Friday at 11:00 AM PDT. > > > > I'd like to go over the latest disk io numbers. > > > > Here is the latest version: > > > > http://www.slac.stanford.edu/~becla/tmp/lsst_diskIO_estimates_v04.doc > > http://www.slac.stanford.edu/~becla/tmp/lsst_diskIO_estimates_v04.xls > > > > > > Shortly speaking the numbers looks very reasonable for > > everything except two temporal queries, and both have the > > same problem (see below) > > > > Phone number: 866 330 1200 > > Pass code: 300 2363 > > > > Jacek > > > > ====================== > > > > Select time series data for given cone: > > > > SELECT o.objectId, o.ra, o.dec, i.MJD, d.* > > FROM Object o, CCDImage i, DIASource d > > WHERE o.ra BETWEEN ra1 + cos(decl1)*(0.5) > > AND ra1 - cos(decl1)*(0.5) > > AND o.decl between decl1 +0.5 and decl1 - 0.5 > > AND o.objectId = d.objectId > > AND d.ccdImageId = i.ccdImageId > > ORDER BY o.objectId, i.MJD ASC > > > > > > Object table: 10 million rows (one partition) > > DIASource table: 45 billion rows > > CCDImage table: 135 million rows > > > > > > At the moment, if I assume a modest 1% selectivity for ra,dec > > that gives 10K rows from Object table. > > > > Join with DIASource (5 DIASources per star) gives 50K rows, > > then join with CCDImage (assuming 10K DIASources per > > CCDImage) gives 50*10^12 rows. That is over a petabyte, which > > now needs to be sorted, then for each row we need to fetch > > full DIASource row.... > > > > > > _______________________________________________ > > LSST-data mailing list > > [email protected] > > http://www.lsstmail.org/mailman/listinfo/lsst-data > > > > _______________________________________________ > LSST-data mailing list > [email protected] > http://www.lsstmail.org/mailman/listinfo/lsst-data ----------------------------------------------------------------------------- Dr. Kirk D. Borne NASA Goddard Space Flight Center, SSDOO Program Manager, QSS Group Inc. and George Mason University, Associate Research Professor, College of Science <mailto:[EMAIL PROTECTED]> Tel. +1-301-286-0696 Fax: 301-286-1771 Staff page: http://rings.gsfc.nasa.gov/~borne/ US Virtual Observatory: http://www.us-vo.org/ Large Synoptic Survey Telescope: http://www.lssto.org/ _______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
