Keywords: DataAccWG
Some thoughts inline.
Deep detection cycle: twice more frequently than release
cycle. Why? Because every few weeks we will have enough
new epochs to make it worthwhile
- deep detection is the largest ingredient of making
data release. Why not call each deep detection run
a 'release'?
Does 'calling it a release' mean it's accessible to the public? If so,
doesn't the corresponding source table need to be archived (increasing
deep storage costs)?
association pipeline assumes there is 1-to-1 (object-to-source)
mapping
- yes, the current algorithm is: take the best match
(closest distance)
--> need to make it clear in user case
- currently in the schema: Source table allows a single source
to point to one Object. Sufficient.
I guess I still have one concern here - if we implement alert generation
by only remembering and looking at the closest match, how do we ever
evaluate whether the closest match was a good choice? Would it be
desireable to (at least for now) make decisions using just the closest
match, but at the same time to include data for all object matches in the
alert? This could either be for alert QA purposes, or perhaps so alert
consumers can look at the stuff we ignored. I'm no astronomer so I can't
judge this, but my thinking is that if one includes small image cutouts
around the region-of-interest, maybe at least including extremely nearby
object data is also useful. If it is useful, then including it up front
would save the interested consumer a latency destroying query to the
object catalog (and maybe some query load at the archive center).
real time updates of Object table at the Base Camp
==================================================
discussion around proposal outlined in:
http://www.slac.stanford.edu/~becla/tmp/ObjectTableUpdatesAtBase.doc
- can't split per filter, association pipeline will
need information related to all filters
- so we have to read info for all filters, but
will write back info for a single filter
- need to use new number of fields: 2500
--> update the doc: redo the calculations
Since paritioning isn't happening per filter anymore, just per field, is
it possible to do some further intra-field partitioning: not so much that
one sacrifices the large block transfer sizes, but enough so that a visit
doesn't entail loading neighbouring fields in their entirety?
providing efficient query access to the Source table
====================================================
...
- mapping table is good because different sources
corresponding to the same object might have very
different positions, and mapping table will allow
them to be clustered together
- if we cluster sources based on objectID + time
we can achieve the same
Just out of curiosity - at the Tucson meeting, I thought it was mentioned
that the equivalent of Sources should/are being stored in tracklets, i.e.
that moving objects essentially get their own catalog and "source" table.
Is this a misunderstanding on my part? Specifically, as part of deep
object processing, is the deep-object pipeline going to go back and
measure a Source for each detection of the moving objects?
Cheers,
Serge
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
