Keywords: DataAccWG Notes from DataAccWG telecon 21 Apr 2006
Attendees: Jeff Kantor, Ray Plante (moderator), Ani Thakkar staging data use case ===================== As part of the email discussion that followed up the introduction of this topic on the Database telecon, Jeff pointed out that we are planning for a 5-Gbs, full-duplex connection between the base and the archive center. On this note, Ray posed the general question, do we need to plan for efficient mechanisms for returning (database) data from the archive center to base facilities (i.e. that minimizes copying data that is either already there or is not needed for the next night of observing), or can we expect to get away with simpler approaches (i.e. just transfer everything we think we need regardless). Upper limit arguments suggest that efficient strategies may not be necessary. If it is a requirement that we transfer all of a night's data (15 TB) in 24 hours, then we should be able to transfer back the new database data from that night, which is less than that (6 TB?). The database data we actually need is likely much less than that. So, not considering the image data requiring caching, we should have more than enough bandwidth. Separate from this are constraints on what gets cached based on the storage capacity at the base facility. Discussion is limited by our understanding of database sizes: (size of records, number of records per patch of sky, etc.). We have some notion of these numbers today, but it is time to collect them together. We need to connect with people working on scheduling issues to see what the requirements are on data staging, and if limits on capacity impose constraints on the scheduling (action: Ray). event database ============== A discussion about an event framework was started last week at the Middleware WG; see docushare Document-1615 for a conceptual design document. This document touches on the event database as a site-local, rolling cache of time-ordered events going back some limited time into the past, primarily for the purposes of debugging. Jeff recommended that a useful contribution of this working group would be to help define the kinds of events and data content required. In particular, we could walk through the UML model to identify event producers and consumers. _______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
