Keywords: DataAccWG
Attendees:
Tim Axelrod
Deborah Levine
Kem Cook
Ani Thakar
Maria Nieto-Santisteban
Sergei Nikolaev
Jacek Becla
object becoming two objects
===========================
Suggested by Robert at the meeting in Tucson:
- keep number of children and parent id for each object
- perhaps we want to keep pointers to children?
- parent/child relation doesn't well reflect LSST use cases
- although there are cases where it helps, e.g.
it helps with proper motion
- in LSST we will have the following cases:
- "confusion that comes and goes". We will switch
back and forth between 1 object and multiple objects
- 1 object will split and stay split
- multiple objects will join and stay joint
- solar moving object will merge with non-moving object,
than would go away
- as our knowledge improves, these use cases will be less
frequent that at the beginning of the survey
- perhaps we should call it confused/resolved instead of
children/parent?
- what is the ration between confused-resolved objects
vs never-confused objects?
- in crowded place, most things might be confusable
- confusion will be different for different bands (by design)
- we will use certain filters in good weather,
and different filters in bad weather
- that introduces nasty systematic errors
--> will discuss it further this Friday
--> Deborah will try to summarize what info related to
confused/resolved objects will be readily needed
from database from a scientist point of view
Data used for testing Object-table partitioning?
================================================
- use data from USNO
- naturally that data will be incompatible with our
prototype schema, so transfer ra, dec, and other
attributes well aligned with our schema, and fill
remaining gaps with randomly generated numbers
- ~ 1 billion objects, 0.5 TB of data, covers entire sky
- Jacek will work with Kem on getting the data
FOV in DC2?
===========
- there will be more than one source of images in DC2
- some data sources will have full LSST FOV images
- some (e.g. sfht legacy) will have smaller images
how to split data from different surveys/skymapper for DC2?
===========================================================
- probably right at the top, so effectively at db level
- no objections, it'd be crazy to mix data from different
surveys
--> so this becomes a baseline from now on
--> we will not be doing any cross correlation between
data from different data sources in DC2
TIMESTAMP as a way to version data?
===================================
- Using TIMESTAMP would mean that the same object would always
have different version at base camp and at archive center
- if it takes more than 1 second to update many objects,
these objects would have different TIMESTAMP (although logically
they should have the same version)
- a more discrete number, like pipeline version or something
like this is more natural, not time when transaction was
executed in db
- this issue is closely related to data provenance
--> defer this topic, provenance will be dealt with in DC3
Is most recent history of objects (and no DIASources)
sufficient to generate alert?
=====================================================
- our current scheme:
- immediately after Deep Det we transfer entire object catalog
from Archive Center to Base, and generate alerts
based on Object table only
- later, we generate alerts based on info from object catalog
and recent DIASources
- in this scheme fidelity of generating alert is different
depending at what point in time alert is generated
- the system has to be flexible (e.g. perhaps what works
at the beginning of the survey will not work later)
- it would be useful for some people who will be doing science
with transients and who deeply care about alerts to tell us
what they are thinking
- maybe raise that at all-hands next Friday
- in general, we need to have a way to get our questions out
to the collaboration and get feedback
- will be discussed next Friday at all-hands (Tim's session)
Big meeting this Friday: 9:00 - 11:00 PST (2 hours, not 3 hours)
thanks,
Jacek
Jacek Becla wrote:
Keywords: DataAccWG
Hi,
We will have a database telecon tomorrow (Wednesday) at 11:00 am PST.
Agenda:
- partitioning Object table (in particular how to test)
- other topics depending on who is present...
Phone number: 866 330 1200
Pass code: 300 2363
Reminder: we are going to have a big 3-hour video conf this Friday,
so let's treat this one lightly.
Jacek
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
