Hi David,
> I am a bit surprised that one needs to store the whole catalog in > memory no matter how it is organized. We don't need to store entire catalog in memory, however LSST will visit ~1/3 of the visible sky every night. So potentially it may need 1/3 of the entire catalog. And unfortunately we will not know well ahead of time which fields will be observed next. > The small chunk needed for a single LSST observation should > be a relatively small and quick > read-only access to a database, and I would have thought that this > could be done in milliseconds, not seconds. Yes, but we have to cross match against up to ~10 million objects per visit (that is how many objects per field will come out of Deep Detection Pipeline). Also we expect to update ~100 K objects in the Object Catalog per visit. These numbers start to add up very quickly =)
So I think that I am getting back to my previous concern about what is to be accomplished by the base camp correlation to an existing catalog. Who is the consumer of this correlation, and what are the needs?
Did you consider calling into the DataAccWG telecon tomorrow? A lot of the issues you are touching upon will be discussed there Thanks, Jacek _______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
