Hi Kem,
Yes, I agree with all your points. On the other hand...there should be
a significant overlap between the information needed tonight and the
information needed tomorrow night. The op-sim output should give us a
good idea of what that overlap factor is, but certainly it should
significantly ease the average amount of data needed by the base each night.
Tim
Kem Cook wrote:
Hi Tim and Jacek,
Just to play devil's advocate, we may need quite a bit more than 6 TB of
template images. While we may only take 1000 images between astronomical
twilights, we may not be able to accurately predict which fields and
filters these will be. Variable conditions may push us to observe fields
which are lower priority. We may also do z and y observing in twilight
conditions which gives us a few hundred more field/filter combinations.
We may also need color information for diffraction correction
calculations. Might we also need our templates to be better than integer
precision?
Baseline with twilight observing:
1200 images x 6.4 GB/image X 0.80 unique area = 6.1 TB perfect planning
x2 for color information
x2 for float
x1.3 for spare field/filters for unpredicted weather/seeing backup
gulp, 31.7 TB (! upper limit, I hope)
Kem
Hi Jacek,
Here's a rough calculation. I assume a typical night of 10 hours,
which corresponds to about 1000 observations of distinct fields (sky
positions). For each of those sky positions, we need at the base:
a) Template images for the fields. At roughly 6 GB/image, that gives us
6TB
b) Recent catalog data for all objects in the fields brighter than about
V=25. How much data that is will vary quite a bit, as a previous email
I sent out explains in some detail. For present purposes, I assume
that there are 150000 objects/deg^2 that qualify. I assume that for
each object we need summary information, which I estimate at 200 bytes
(very rough), and the latest 10 epochs of measurements at 50 bytes each.
This then gives us 1000 fields * 150000 * 10sq deg * (200 + 10*50) =
1.5TB. I'd say at present this is no better than a factor of two
estimate.
c) calibration images - this is maybe 100GB, negligible.
So a rough guess is 7.5TB.
Cheers,
Tim
Jacek Becla wrote:
Jeff/Tim/Ray
I'd like to start thinking about details of pre-staging
data at the base / partitioning it, but before I do that
I need to understand how much data we are talking about
per night (the question was brought up at today's Database
telecon).
If you could find that out and give me even a rough estimate
whether it is a terabyte or a hundred terabytes, that
would help a lot.
Thanks,
Jacek
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
