On 3/23/2012 11:31 AM, Phil Stracchino wrote: > To return somewhat to the original issue, if I ever get enough > uninterrupted free time on my hands, I have a thought to looking at the > spooling problem myself. I have a design approach in mind already. > This is the basic process as I see it: > > > Configuration items - > - Spool area location > - Spool area high-water mark (preferably as minimum free space) > - Spool file size > > Let's say for sake of example that there is 25GB available in the spool > area, and we set the spool high-water mark to 20GB and the spool file > size to 1GB. Each running job will write to a spool file in the spool > area until that file reaches 1GB in size or the client has sent all of > its data, whichever comes first. It will then close that spool file, > marking it available for despooling, and begin writing a new spool file > if required. No job will START a new spool file if the spool area > reaches the high-water mark, but jobs can *continue to write* to already > open spool files as long as space remains. If the spool area becomes > full, then running jobs will be paused until space becomes available. > > Meanwhile, the device writer sits and waits for a spool file to be > marked as available. It maintains a queue of spool files to be written, > queued in the order they became available. Any time one or more spool > files is present in its queue, it opens the top spool file, writes it > out to the device in a single streaming write, then closes it and > deletes it. It then opens the next spool file on the queue, writes that > to storage, and deletes it in turn. It continues to despool and delete > spool files until it runs out of available spool files, then pauses > until more spool files are marked available. > > > This approach: > - Allows most jobs to be spooled and despooled simultaneously, after the > first gigabyte (or whatever the spool file size is); > - Allows multiple jobs to spool simultaneously; > - Minimizes the time the storage device spends waiting for spooled data > to write; > - Should not significantly increase total backup time over an unspooled > configuration.
A FIFO queue makes sense, but how will SD know when it has completed writing all of the spool files for a particular job? ------------------------------------------------------------------------------ This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
