Hello, This is something that we have been "planning" for some time, and it is actually in the projects file as "step 2" of item 14, but I have added it as a project on its own.
Thanks, Kern On Tuesday 17 November 2009 21:50:53 Jesper Krogh wrote: > Item n: Concurrent spooling and despooling withini a single job. > Date: 17 nov 2009 > Origin: Jesper Krogh <jes...@krogh.cc> > Status: NEW > What: When a job has spooling enabled and the spool area size is > less than the total volumes size the storage daemon will: > 1) Spool to spool-area > 2) Despool to tape > 3) Go to 1 if more data to be backed up. > > Typical disks will serve data with a speed of 100MB/s when > dealing with large files, network it typical capable of doing 115MB/s > (GbitE). Tape drives will despool with 50-90MB/s (LTO3) 70-120MB/s > (LTO4) depending on compression and data. > > As bacula currently works it'll hold back data from the client until > de-spooling is done, now matter if the spool area can handle another > block of data. Say given a FileSet of 4TB and a spool-area of 100GB and > a Maximum Job Spool Size set to 50GB then above sequence could be > changed to allow to spool to the other 50GB while despooling the first > 50GB and not holding back the client while doing it. As above numbers > show, depending on tape-drive and disk-arrays this potentially leads to > a cut of the backup-time of 50% for the individual jobs. > > Real-world example, backing up 112.6GB (large files) to LTO4 tapes > (despools with ~75MB/s, data is gzipped on the remote filesystem. > Maximum Job Spool Size = 8GB > > Current: > Size: 112.6GB > Elapsed time (total time): 46m 15s => 2775s > Despooling time: 25m 41s => 1541s (55%) > Spooling time: 20m 34s => 1234s (45%) > Reported speed: 40.58MB/s > Spooling speed: 112.6GB/1234s => 91.25MB/s > Despooling speed: 112.6GB/1541s => 73.07MB/s > > So disk + net can "keep up" with the LTO4 drive (in this test) > > Prosed change would effectively make the backup run in the "despooling > time" 1541s giving a reduction to 55% of the total run time. > > In the situation where the individual job cannot keep up with LTO-drive > spooling enables efficient multiplexing of multiple concurrent jobs onto > the same drive. > > Why: When dealing with larger volumes the general utillization of the > network/disk is important to maximize in order to be able to run a full > backup over a weekend. Current work-around is to split the FileSet in > smaller FileSet and Jobs but that leads to more configuration mangement > and is harder to review for completeness. Subsequently it makes restores > more complex. > > > > --------------------------------------------------------------------------- >--- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 > 30-Day trial. Simplify your report design, integration and deployment - and > focus on what you do best, core application coding. Discover what's new > with Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Bacula-devel mailing list > Bacula-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-devel ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
