Hi Ray, A few comments on your comments that probably need commenting below:
Jeff > From: Ray Plante <[EMAIL PROTECTED]> > Reply-To: Ray Plante <[EMAIL PROTECTED]>, LSST Data Management > <[email protected]> > Date: Tue, 4 Apr 2006 21:31:10 -0500 (CDT) > To: LSST Data Management <[email protected]> > Subject: [LSST-data] Changes to Framework Domain model [MiddleWG] > > Modelers, > > I've integrated the pipeline framework classes from the sandbox into the > Framework Domain Model package. In particular, I modified the Processing > Framework Summary diagram. As part of this integration, I deleted some > classes--a few from the package completely, others only from the diagram. > One reason for keeping some in the latter group was because they contained > information--attributes and operations--that may need to get migrated over > to other classes; I didn't want these forgotten, though I'm not sure at > this point what/how these need to be migrated. Others I kept around > because they are referred to prominently in Use Case models; I thought I > would keep them around until those Use Cases can be updated. > > In particular, here are some issues that need further attention by our > skunk team: > > 1. Component > > This class figures prominently in the Pipeline Construction System use > cases. It's existance was based on the idea that components would be > assembled to create Processing Steps, which in turn would be assembled > into Pipelines. We discussed in the current model the idea that pipeline > developers create Processing Stages as the reusable unit for assembling > pipelines. So do we still need a Component class? > > We still will have the need for things like an Image class which Stage > developers would utilize. Is this a Component? > > If we need this concept, then we need a crisp definition; otherwise, we > should delete it. In either case, the PCS use cases will need to be > updated accordingly. I see this as a PCS issue. > I agree. If it is still useful, it is in the context of the process of creating a Processing Stage. Regarding the Image Class, we had been thinking of Components as Processing units, rather than data managing objects like the Image class. I think it is an interesting idea to generalize the definition of Component to any useful framework class employed in creating a Stage or Pipeline, but only if we need to create behavior or attributes at this level of abstraction (e.g. Be in a Component Library) > 2. Provenance > > The Provenance model needs a pass in the new light of the Pipeline model. > (Perhaps Jacek has already started this.) In particular, since Processing > Step has been deprecated, we need to deprecate Processing Step Provenance. > The schema would need to be re-arranged accordingly. > Yes > 3. Job > > We have used this word a lot in the discussion of the model; however, it > does not currently have a formal role. Do we need this as a class? If > so, it needs a crisper definition. I am inclined to delete it and > recreated it if necessary. The Pipeline Use Cases need to be updated > accordingly. This one falls on my plate, I think, but your comments are > welcome. > I still have the idea that the entire workflow of the system and it's current state (pipelines running, data being ingested, data being transferred, etc.) is somehow visible and being monitored. It seems to me that there is still room for a "running" pipeline or a data ingest or transfer proces in it's totality as a job in this context. I think we need to hit this hard (probably after DC1) and cover the Events, Event Channels, Job Monitoring, etc. aspects. I don't think we've really gotten into this area deeply enough. > cheers, > Ray > > _______________________________________________ > LSST-data mailing list > [email protected] > http://www.lsstmail.org/mailman/listinfo/lsst-data > _______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
