Don,
Sorry to miss your telecon. I wanted to attend to give you
some heads up that one of my main worries now is CPU to support
query load. I fear we are seriously underestimating how much CPU
power we will need in order to support the hypothetical query load.
To illustrate the problem, consider this:
- if we keep major indexes in memory (20 TB), we need 4K disks
- if we don't keep them in memory, we need over 1 BILLION disks.
That shows how much we use the indexes (select rows, join) to
answer these queries. I eased the disk IO problem by keeping
most frequently accessed data in memory, but we will still have
to access these multi-billion-row indexes over and over.
I am planning to do a very rough estimate how much CPU
we would need to support the hypothetical query load in
the next few days.
Jacek
Don Dossa wrote:
From the tech assess concal -7-17-2006
The discussion centered around two main topics: Archive computing needs
and network bandwidth between the archive site (AC) and data access
center (DAC).
We assume the astronomical image pipelines are not drastically changing
but the query load imposed on the archive and data center sites can be
variable and require additional computations, disk IO, and network
bandwidth.
Given as base computing requirement of 25 TF, then we assume an archive
compute requirement of 90 TF of which about 80 TF should be available.
This is allocated as 25 TF to reprocess the nightly raw images which are
delivered over 24 hours. An allocation of 25 TF is for on-demand
reprocessing but some of this load could be shifted to the researcher's
home institution. Another 5 TF is needed for deep detection
processing, and 5-10 TF to handle queries. The remaining 25+ TF is used
to reprocess years of data. This personally seems low since that
implies 3-4 nights of images are done per day. Other than the
processing of each night’s images, we expect the compute requirements to
constantly increase over time.
For our estimating purposes, we assume that approximately 50 low data
volume requests require about 1/2 GByte of data to be returned over ~ 10
seconds and high data volume requests of 20 requests needing about 6
GBytes each returned over ~ 60 minutes.
Hello all,
Regarding the Archive Center and Data Access Center network
requirements, it appears we have the following identified requirements:
- 10Gbps dedicated connecting the AC and DAC in support of data
replication (bulk transfers)
- 10Gbps from each center to the external research community for queries
(aggregate of low volume and high volume queries as defined in our
sizing model)
It sounds as if both NCSA and SDSC expect to be able to support LSST
with these bandwidths in the 2013 time frame. That is our planning
assumption.
Further, it appears that there are multiple network initiatives that
might be used to provide this bandwidth, including NLR, I2/Abilene, and
TeraGrid. I note also, that these are not mutually exlusive entities,
i.e. I2 is a founding member of NLR. My question is:
For the 2 paths above, given the institutional memberships of NCSA and
SDSC in these networks, what is the likely source of the bandwidths we need?
Jeff
Hi folks
I just want to confirm that our discussions with our network expert here
at NCSA paints a projection similar to what has already been discussed.
In particular:
o the current pipe to the community (not TeraGrid) is 1 GB/s
o the expected evolution is next 10 GB/s, then 40 GB/s
o a 10 GB/s is in the works for both NCSA and the University campus
o it's not clear when 40 GB/s might come around; it's possible that it
will not be in place for first light.
Overall, I think it is fine to assume that we will have the bandwidth
needed to support our usage model.
cheers,
Ray
Jeff,
My sources tell me not to rely on I2 as it currently is, to be present
in a few more years, I don't want to start any nasty rumours but that's
the word I get. NLR seems to the likely candidate for the long term. I'm
not sure that I2 technically speaking was a "founding" member or NLR but
are a consortium that bought two shares for $10M for HOPI.
NLR have multiple fibres and bandwidth but my understanding is that only
4 are lit at present, each for different services. Interestingly if one
looks at todays weathermap for level 2 and 3, they are very underutilised.
http://weathermap.grnoc.iu.edu/NLR_Layer2_jpg.html
http://weathermap.grnoc.iu.edu/NLR_Layer3_jpg.html
and Abilene for comparison.
http://weathermap.grnoc.iu.edu/abilene_jpg.html
Correct me if I am wrong but TerraGrid I think. is relating to Terrabyte
SAN storage and Machine TerraFlops not necessarily Terrabit bandwidths,
at least that is my limited understanding. I stand to be corrected by
TerraGrid users.
ron
------------------------------------------------------------------------
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
