On Thu, 5 Oct 2006, Russell E Owen wrote: > From something Jacek said yesterday (but not from any email I've > seen) it sounds as if you've managed to create 6300 different > database tables, rather than 192 tables containing 32 files each. Is > this the case? If so, something needs to be fixed (either the policy > files or dbingest itself). Creating a new table for each file would > slow things down -- though how much, I would not care to guess.
I'm not sure that we are creating 6300 different database tables, unless that is built into the dbingest service. We just write the files to the ingest directory, and the service slurps them up. > I could easily modify dbingest so that it processes all new files for > a given slice before checking the next slice. But if each image needs > significant time to be processed (as I assumed when I designed > dbingest) then I doubt this would have *any* effect at all during a > normal run. Agreed. > It certainly could help if you started dbingest well > after data was being pumped out by the pipeline (such that there was > a collection of data files already waiting to be ingested). It becomes a question of when we need the data in the database. Right away, I think, particularly, if the correllation with the standard stars for WCS determination is done in the database. Should we schedule a quick telecon today, say 1pm PT, to talk through this? The people we should have on-line at least should be, Jeff Kantor Jacek Becla Russell Owen Ray Plante Greg Daues cheers, Ray _______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
