Keywords: DataAccWG
Hello,
1)
I found one mistake in the original estimate for non-split Object table.
Fixing the problem brought down the number of disk heads:
- indexes not in memory 166K --> 140K
- indexes in memory: 44K --> 16K
2)
I redid the disk io spreadsheet assuming non-split Object table
and measurements for Galaxies per frame.
- indexes not in memory: 208K
- indexes in memory: 18K
That is "only" less than 50% worse comparing to
the non-split object table. (especially if we put index in memory).
3)
I can bring down the numbers even further, e.g. down to 6K disks
by choosing a larger data block (512K).
All the numbers below assume disabled "select time series data
for a given cone" query (this query can still drive the numbers
unbelievably high, even with all indexes in memory)
I need few more days to better validate everything, but I know
some of you really want the numbers ASAP, so there they are,
a little better version than the one before.
Happy 4th of July!
Jacek
Jacek Becla wrote:
Hi Jeff
I promised to give you the disk io estimates by the end of Friday.
I think I am slowly getting there, there is one query (classified as
"low volume"!) that is causing troubles: "select time series data
for a given cone". It highly depends on index selectivity, and
if we choose a reasonable selectivity (50% of one partition is
very reasonable, but even few % is causing problems), we end up
fetching many millions of data rows which drives the number
or required disks up many times (how many times - depends
what selectivity we pick).
If I pick very, very low selectivity for that query (effectively
disable it) the spreadsheet suggests we need 166 K disk heads
assuming we want to run all 300 low volume queries and 10 high
volume queries, no super high volume query. (Reminded, the
spreadsheet is for DR2).
We can drive the numbers down by choosing to run less
queries concurrently (in the current model all low volume
queries will be done after 10 sec and all high volume
after 10 min).
It should also be possible to drive the io down by tuning
the queries.
I assumed disk random reads, block size 64K, realistically
for a system with hundreds of simultaneous queries we should
not count much on nice sequential reads.
General comment:
- none of the queries use Source table, they use DIASource
table instead. DIASource is 5% of the Source table, so
if someone chose the Source table, the numbers would get
much worse. I don't feel comfortable with it: why did we
reserved 100s of TB for Source table in the Database
Size Estimations if we don't have any queries that would
use that?
- the numbers will get (much) worse if I redo the spreadsheet
using the latest sizes (non-split object table...)
Here is the most recent version of the spreadsheet and the doc:
http://www.slac.stanford.edu/~becla/tmp/lsst_diskIO_estimates_v04.xls
http://www.slac.stanford.edu/~becla/tmp/lsst_diskIO_estimates_v04.doc
It will take me few more days to clean up the documentation
and QA the spreadsheet better.
I guess we knew the numbers would not be pretty....
To put a positive spin on all this, I think there is a solution:
if we purchase 2.5 TB of RAM (very reasonable in 2012), we can
pin all indexes used for these queries in memory, which will drive
the number of required disk heads down to 44 K. BTW, this still
does not solve the problem with time-series query which wants
to touch petabytes of data rows... If this idea sounds attractive
(think "PetaCache" =), I will do a more complete estimate
of RAM needed. I did a first sketch in the spreadsheet,
and you can turn it on and off to see the effect on various
queries.
Jacek
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
