Maria,
Thanks for clarifying - some further comments are inline.
On Mon, 04 Dec 2006 18:03:17 -0800, Maria A. Nieto-Santisteban
<[EMAIL PROTECTED]> wrote:
Serge,
The more realistic distribution you have of the objects in the sky, the
more realistic distribution you have of the objects on the disk. Doing
the
10, 20 arcsecs and even the individual perturbation can give you funny
distributions. A better aproach would be (I think) to break the sky in
regions, make an histrogram to count number of sources, then generate
randomly the right proportion of point for each region. Probably
have into account the fact that USNOB probably see as many stars as LSST
in the Galactic plane, but LSST will go deeper out of the galactic plane,
which means the density will be higher.
This is good to know - I've never actually tried the perturbation method
to see what the results look like, it just seemed "good enough". Well...
at least better than just assuming worst case density across the whole
strip :).
Anyway, I think maybe we should test all the way through x-match. Maybe
not the x-match itself, but at least up to the part where we create the
in-memory zone-ra index just prior to x-match. I may very well be wrong,
but it seems to me that the layout of things on disk might influence how
fast we can do this in memory.
To be honest I'm just don't see a strong argument for 2kb rows vs. 200
byte rows or vice versa. As far as I can tell either approach gives us a
basis for comparison. What about doing both?
The row size makes a difference. The samller is the object the more can
fit in a page and the lower the I/O. A 64k page can host about 30 ~2Kb
objects vs 300 200b objects. The smaller the number of objects per page
the higher the cost.
Yes of course you are right, but - in the 200b row case we will be
reading/updating/inserting 10x more objects (using some kind of random
object generation or 10 perturbed copies scheme), so the number of pages
read/written from disk should be about the same. If I understood Jacek
correctly, he basically wants to max out the test platform - i.e. use as
many 2kb rows in 3 strips as we have the space to store, or use realistic
row count (which means we have to shrink to 200b per row to fit all-data
on disk).
Cheers,
Serge
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
