Kem: gee, I would hope that those kinds of queries are not precluded, since those are likely to be the most "interesting" (novel) events, which ought to be broadcast to the world ASAP.
- Kirk > Date: Fri, 05 May 2006 14:34:30 -0700 (PDT) > From: Kem Cook <[EMAIL PROTECTED]> > Subject: Re: [LSST-data] Base Camp Database requirements > To: LSST Data Management <[email protected]> > > Hi, > > So, we may not need all the static data, but I thought that it would be > value added to be able to say that a new transient occured at the position > of a known, but non-variable object (say a distant, barely resolved > galaxy), or that a new transient appeared x arcsec offset from the > nucleus of a rather well resolved galaxy. or that there was a transient at > the position of a known M dwarf (perviously non-variable). > > Kem > > > Hi, > > > > A few thoughts following our telecon: > > > > Jacek Becla wrote: > > > >> Keywords: DataAccWG > >> > >> Hi, > >> > >> I just had a quick word with Kem, here is the summary plus > >> some estimates/comments I added. > >> > >> > >> 1) What is the expected size of pre-cached archived data > >> at the base camp? > >> We are expecting to have roughly 100TB of image data, > >> corresponding catalog data will probably be ~10% of that > >> So it is ~10TB (upper bound). > >> > >> 2) How much of that data will be needed on average per image? > >> ~5GB (upper bound) > >> > > I think we need a combination of deep object data, which does not have > > time history (it is constructed from deep stacks of images over a long > > time span), and shallower object data which does have time history. For > > the deep object data, if we assume 50 galaxies per arcmin**2, we have > > about 2e6 objects per image. We need to work on the object data > > definition, but let's suppose that for each filter we keep three > > different kinds of magnitudes and their associated errors. We will > > have some kind of classification, and a variety of data quality flags. > > There will also be detailed shape information, but we likely don't need > > this at base, so I ignore that here. If each of those mags/errors is a > > 32 bit float, the flags take 16 bits, and the classification 8 bits, and > > we have 5 bands (U will not go deep), then we need 2e6*5 filters*(3 > > mags*2 quantities/mag * 4 bytes/quantity + 3 bytes for flags) = 2.7 GB. > > > > The time dependent data will basically be limited to stars at roughly > > 25th in V. The density on the sky is highly variable, but an average > > value of about 2 per arcmin**2 is probably not too far off. This gives > > us roughly 1e5 stars per image on average. I think we will need summary > > information for each star, plus perhaps the most recent ten measurements > > (in whichever bands we have them). Again, we suffer from not having > > defined our object data well. But suppose we summarize the star with > > 10 floats per band, and then have for each time point 2 floats plus 2 > > bytes of quality flags. Then for each star we need 5*10*4 + 10*(2*4 + > > 2) = 300 bytes. The total for stars is then 0.07 GB. > > > > So, given the assumptions about what we need, I certainly think that > > Kem's number is in the right ballpark. However, I wonder if we really > > need to prestage all this data for every image. After all, we are > > operating on difference images at base, and even granted that the > > detection are supplemented by some list of objects to always measure, we > > will be needing object information for only a tiny fraction of the stars > > in the image - perhaps a few percent. So, we need to fetch info for a > > few * 1e3 objects from the total of 2e6 total (mostly deep) objects for > > which we have data. Does this change our strategy? > > > > Tim > _______________________________________________ > LSST-data mailing list > [email protected] > http://www.lsstmail.org/mailman/listinfo/lsst-data ----------------------------------------------------------------------------- Dr. Kirk D. Borne NASA Goddard Space Flight Center, SSDOO Program Manager, QSS Group Inc. and George Mason University, Associate Research Professor, College of Science <mailto:[EMAIL PROTECTED]> Tel. +1-301-286-0696 Fax: 301-286-1771 Staff page: http://rings.gsfc.nasa.gov/~borne/ US Virtual Observatory: http://www.us-vo.org/ Large Synoptic Survey Telescope: http://www.lssto.org/ _______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
