Thanks for the response. I am referring to this page in the documentation: http://www.bacula.org/en/dev-manual/main/main/Data_Spooling.html
I believe the above page is inline with what you say below. I am running 5.2.6 on Centos 5.8 with a tape library with 2 drives. When I have launched jobs together, meaning, when I launch a set of jobs in quick successtion so that they all vie for tape device resources, and as those are assigned, they begin spooling, I have observed the behaviour described in the web page above and your email below. The case where I have not is this: Day 1, launch set of jobs which occupy all drives, and use Data Spooling. These run as expected and smaller/faster jobs complete sooner. A few jobs (say two large slow ones to emphasize the situation) remain spooling for longer, but eventually reach their despooling phase, at which point they both occupy one of my two tape drives. This situations proceeds into Day 2. Day 2, launch a different set of jobs, and use Data Spooling. The jobs from the previous day are still despooling, and occupying all available drives. And there is plenty of spool space available. All jobs in this example have the same priority, level, use the same pool, to which the tapes in the occupied drives belong, and there are no concurrencies being exceeded. What I see for these new jobs, is that they do NOT begin to spool. There status in bconsole is: (for Director) 12345 Full JOBNAME.0000-00-00_00.00.00_00 is running (for Storage Daemon) spooling=0 despooling=0 despool_wait=0w [I don't seem to have captured what they say in the device field.] They remain in this state until the jobs from the previous day finish despooling and free up their drives, at which point the new jobs begin spooling. After doing some reading, I wondered if it's not possible that they are stuck in the state before despooling, which I believe is checking to see that there will be a tape device and volume available to them when the are ready to despool. It looks from my view, that once a job is started it can spool to it's heart's content, even if all drives are occupied by other jobs that are despooling, but that jobs cannot 'start' if all drives are occupied by other jobs that are despooling. I can gather whatever evidence you need on this or open a bug if that's appropriate. This is causing some major grief, as we were getting better results with tiny spool files. I think in that situation, the problem I describe still exists, but chances are there will be a moment when all existing jobs are in a spooling, rather than despooling, state, at which point new jobs can jump in, get a drive assigned, and start spooling. However, in that case, we can't of course make use of the benefits of having spool files sizes as large or larger than our jobs. thanks, Stephen On 3/21/12 2:27 AM, Kern Sibbald wrote: > Hello, > > Can you give me the exact location of that text in the document and > what Bacula version you are referring to? > > The facts as we know them, including the design of the software, and what > we have seen from testing are: > > - When a Job begins despooling, that Job and only that Job stops > spooling and begins despooling. > - When the Job is done despooling, it will begin spooling again. > - While any Job is despooling, all other Jobs will continue spooling > providing there is sufficient spooling space. > - If Jobs run out of spooling space, they start despooling, but only > one job at a time despools to any given volume, so they may wait. > - Once the spooling space fills, if you are writing to several Volumes, > you can get into a sort of spooling thrashing situation where various jobs > finish despooling and others start using the space liberated. Thus it > is important not to run out of spooling space. > - Note, when I speak about running out of spooling space, in the terms > I am using those words, it can never happen to a single job. It happens > only when running multiple jobs to multiple devices. > > We have a project planned for Bacula Enterprise where a Job is blocked > because it is despooling will be able to continue spooling providing there > is sufficient space. But this is an Enterprise feature, and unless someone > from the community implements it, it will only be in the Enterprise > version. > In fact, we have a series of plans for adding high end Enterprise features > to the Storage daemon. Most of them should be completed by the end of > the year (hopefully before). > > Best regards, > Kern > > On 03/20/2012 10:46 PM, Stephen Thompson wrote: >> >> Hello, >> >> I was wondering if anyone could confirm what I've noticed on my own >> instance of bacula, which seems contrary to the Bacula manual. >> >> From DataSpooling section: >>> If you are running multiple simultaneous jobs, Bacula will continue >>> spooling other jobs while one is despooling to tape, provided there >>> is sufficient spool file space. >>> This seems to be true, only if the jobs in question were launched at the >>> same time/concurrently. New jobs launched while a job is despooling, >>> are launched into a "running" state, but they do not begin to spool >>> until the existing job(s) finish despooling. >> This is very sad, because I just came into a windfall of spool space and >> I was hoping to run jobs back to back, such that while one set of jobs >> were despooling, I could have the next set spooling, and so on. >> Note, all jobs use the same Pool and have the same priority. >> >> I noticed this because I've increased the spool file size to the size of >> my largest job. Now, rather than before (with smaller spool file sizes) >> when the new jobs had a chance to start spooling while the existing jobs >> were in a spooling phase, the new jobs cannot being spooling apparently >> because the existing jobs stay in a despooling phase, which can be for >> quite some time. >> >> I see an old bug 0001231 with a similar issue, which in the history it >> is pointed out that it may not be that new jobs can't spool while >> existing jobs despool, but that the new jobs cannot verify that they >> will have tape access, which is a step before spooling begins. >> >> I wonder if this is simply the state of affairs and if so, are there are >> any plans to improve upon this 'inefficiency'. >> >> thanks! >> Stephen -- Stephen Thompson Berkeley Seismological Laboratory step...@seismo.berkeley.edu 215 McCone Hall # 4760 404.538.7077 (phone) University of California, Berkeley 510.643.5811 (fax) Berkeley, CA 94720-4760 ------------------------------------------------------------------------------ This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
