Hello all, One comment regarding ingest:
I think that most pipelines will want to handle their data as floats rather than ASCII, while the ingest for MySQL apparently works best with ASCII. I maintain that the pipelines should carry the data in floating point (as they naturally want it) and any conversion to ASCII should occur on the Data Access Framework/DBMS loader side of the interface. I think this also insulates the pipelines from change in case we use another dbms that is more efficient for floats. Jeff > From: Ray Plante <[EMAIL PROTECTED]> > Reply-To: Ray Plante <[EMAIL PROTECTED]>, LSST Data Management > <[email protected]> > Date: Wed, 15 Mar 2006 14:46:06 -0600 (CST) > To: LSST Data Management <[email protected]> > Subject: [LSST-data] Minutes 2006-03-15 [MiddleWG] telecon > > Middleware Working Group Minutes > 2006-03-15 10am PT > > Attendees: Russell, Jeff, Tim, Ray, Marcus, Jacek, Arun > Agenda: > o business > + chair > + the video option > o updates on high-level review of the model > + database ingest > + policy objects > + ... > o identifying areas for further modeling > > Ray requested we have ready access to the UML model via the terminal > server as well as to this "problem statement": > > http://www.lsstmail.org/pipermail/lsst-data/2006-March/001007.html > > 1. Business > > 1.1. Chair > > Due to other demands, Ghaleb asked if he could step down as chair of > this working group, and Ray has agreed to take over. He'll try to get > out agendas ahead of time so non-regular attendees can decide whether > to call in. Suggestions for input to the agenda will be taken at > anytime, including at the start of the meeting; however, ahead of time > is certainly more helpful to everyone. > > Action: Ray: review Jeff's edits to the MiddleWG charter in > docushare. > Comments from others are welcome, too. > > 1.2. Connecting via polycom > > Jeff is encouraging people to access telecons via polycom video when > convenient as a way to reduce costs. This requires that someone from > LSSTC connect the phone into the polycom so that those without polycom > still attend. > > We determined that enough people in the WG to make it worth supporting > video. Jeff and Tim will be our LSSTC person responsible for patching > in the phone. If on a given week, neither are able to attend, > everyone will revert to the phone. > > Action: Ray: arrange a reservation with Iain; send out connection > details to WG. > > 2. UML Modeling > > Jeff introduced this discussion with some guidance on further modeling > through to the "skunk works" workshop. As mentioned in Ray's problem > statement, areas of focus can be an aid to the design of DC1. Thus, > the important areas to think about are: > > o pipeline construction > o pipeline execution > o access to data by pipelines > -- staging input data > -- ingesting output data > > We plan to push Use Cases from these areas down into Robustness models > using entities from the Domain Model. We currently have Domain > objects for Framework concepts (e.g. Image); however we are light on > interface objects, e.g. data ingest service. So we need to reflect > more these later type of nouns in the Domain model. These should be > stereotyped "interface". > > Tim recommend that if we can pick and make progress on some very > specific operations--e.g. executing a pipeline that does > flat-fielding, we would accomplish a lot. > > There was the question of how to approach modeling something this > specific (wherein our current use cases are very general). We > concluded that we create a robustness diagram for the very specific > use case. We might then later create a second version that is more > general. > > 2.1. Review of the state of the model at the high-level > > Ray briefly reviewed the discussion on the Policy class, highlighting > two points: > o the proposal for using Policy objects for transfering > dynamic information between components may be replaced with > something more natural to the Processing Step framework we > choose to support glueing components together. > > o Creating, tweaking Policy objects will be at the center of > "targeting" a pipeline to a data collection and a platform. > > Jacek briefly reviewed the conclusions from the Data Ingest discussion > and doesn't see any major outstanding issues. Ray wants to also bring > up the related use case of "staging database data". > > 2.2. Modeling Plans > > Marcus suggested and it was agreed that the skunk works attedees > should try to generate one robustness diagram prior to next week so > that we can get used to the mechanical and process issues of doing > so. > > Action: Ray, Marcus, Jacek: post to the list today a brief statement > of the robustness diagram we will work on this week. > > Action: Ray, Marcus, Jacek: begin work on a robustness diagram for > discussion next Wed. > > Marcus will do something on targeting a pipeline for some part of > calibration. Jacek will do one on catalog data ingest. Ray will do > one on executing a configured pipeline. We will work these diagrams > inside the Middleware Use Cases folder. > > > _______________________________________________ > LSST-data mailing list > [email protected] > http://www.lsstmail.org/mailman/listinfo/lsst-data > _______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
