Ray,
Would it be simpler to just replicate the ingest service on different nodes, each one monitoring a different input directory? The configuration of the converter pipeline stage would just indicate which directory to write its output to. Since we only need 2 (rather than 200) ingest nodes, half of the converters would write to one directory, and the other half to the other directory.
Yes, it would be simpler and it is probably fine if we don't implement the higher level CIS for DC1. However, in real production the CIS will be responsible for ingesting data from many different sources (e.g. nightly ingest, data from reprocessing, data from simulation, provenance), as well as orchestrating some tasks (e.g. correlating near neighbors). I believe (and Jeff does too) it would be useful to have one CIS that centrally coordinates all these activities and ingest related hardware. Jacek _______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
