On 3/21/12 8:02 AM, Stephen Thompson wrote: > > > Thanks for the response. > I am referring to this page in the documentation: > http://www.bacula.org/en/dev-manual/main/main/Data_Spooling.html > > I believe the above page is inline with what you say below. > > I am running 5.2.6 on Centos 5.8 with a tape library with 2 drives. > > When I have launched jobs together, meaning, when I launch a set of jobs > in quick successtion so that they all vie for tape device resources, and > as those are assigned, they begin spooling, I have observed the > behaviour described in the web page above and your email below. > > The case where I have not is this: > > Day 1, launch set of jobs which occupy all drives, and use Data > Spooling. These run as expected and smaller/faster jobs complete > sooner. A few jobs (say two large slow ones to emphasize the situation) > remain spooling for longer, but eventually reach their despooling phase, > at which point they both occupy one of my two tape drives. This > situations proceeds into Day 2. > > Day 2, launch a different set of jobs, and use Data Spooling. The jobs > from the previous day are still despooling, and occupying all available > drives. And there is plenty of spool space available. All jobs in this > example have the same priority, level, use the same pool, to which the > tapes in the occupied drives belong, and there are no concurrencies > being exceeded. > > What I see for these new jobs, is that they do NOT begin to spool. > There status in bconsole is: > > (for Director) > 12345 Full JOBNAME.0000-00-00_00.00.00_00 is running > > (for Storage Daemon) > spooling=0 despooling=0 despool_wait=0w > [I don't seem to have captured what they say in the device field.] > > They remain in this state until the jobs from the previous day finish > despooling and free up their drives, at which point the new jobs begin > spooling. > > After doing some reading, I wondered if it's not possible that they are > stuck in the state before despooling, which I believe is checking to seew
sorry, correction: "in the state before spooling" > that there will be a tape device and volume available to them when the > are ready to despool. > > It looks from my view, that once a job is started it can spool to it's > heart's content, even if all drives are occupied by other jobs that are > despooling, but that jobs cannot 'start' if all drives are occupied by > other jobs that are despooling. > > I can gather whatever evidence you need on this or open a bug if that's > appropriate. > > This is causing some major grief, as we were getting better results with > tiny spool files. I think in that situation, the problem I describe > still exists, but chances are there will be a moment when all existing > jobs are in a spooling, rather than despooling, state, at which point > new jobs can jump in, get a drive assigned, and start spooling. > However, in that case, we can't of course make use of the benefits of > having spool files sizes as large or larger than our jobs. > > thanks, > Stephen > > > > > > > > > On 3/21/12 2:27 AM, Kern Sibbald wrote: >> Hello, >> >> Can you give me the exact location of that text in the document and >> what Bacula version you are referring to? >> >> The facts as we know them, including the design of the software, and what >> we have seen from testing are: >> >> - When a Job begins despooling, that Job and only that Job stops >> spooling and begins despooling. >> - When the Job is done despooling, it will begin spooling again. >> - While any Job is despooling, all other Jobs will continue spooling >> providing there is sufficient spooling space. >> - If Jobs run out of spooling space, they start despooling, but only >> one job at a time despools to any given volume, so they may wait. >> - Once the spooling space fills, if you are writing to several Volumes, >> you can get into a sort of spooling thrashing situation where various jobs >> finish despooling and others start using the space liberated. Thus it >> is important not to run out of spooling space. >> - Note, when I speak about running out of spooling space, in the terms >> I am using those words, it can never happen to a single job. It happens >> only when running multiple jobs to multiple devices. >> >> We have a project planned for Bacula Enterprise where a Job is blocked >> because it is despooling will be able to continue spooling providing there >> is sufficient space. But this is an Enterprise feature, and unless someone >> from the community implements it, it will only be in the Enterprise >> version. >> In fact, we have a series of plans for adding high end Enterprise features >> to the Storage daemon. Most of them should be completed by the end of >> the year (hopefully before). >> >> Best regards, >> Kern >> >> On 03/20/2012 10:46 PM, Stephen Thompson wrote: >>> >>> Hello, >>> >>> I was wondering if anyone could confirm what I've noticed on my own >>> instance of bacula, which seems contrary to the Bacula manual. >>> >>> From DataSpooling section: >>>> If you are running multiple simultaneous jobs, Bacula will continue >>>> spooling other jobs while one is despooling to tape, provided there >>>> is sufficient spool file space. >>>> This seems to be true, only if the jobs in question were launched at the >>>> same time/concurrently. New jobs launched while a job is despooling, >>>> are launched into a "running" state, but they do not begin to spool >>>> until the existing job(s) finish despooling. >>> This is very sad, because I just came into a windfall of spool space and >>> I was hoping to run jobs back to back, such that while one set of jobs >>> were despooling, I could have the next set spooling, and so on. >>> Note, all jobs use the same Pool and have the same priority. >>> >>> I noticed this because I've increased the spool file size to the size of >>> my largest job. Now, rather than before (with smaller spool file sizes) >>> when the new jobs had a chance to start spooling while the existing jobs >>> were in a spooling phase, the new jobs cannot being spooling apparently >>> because the existing jobs stay in a despooling phase, which can be for >>> quite some time. >>> >>> I see an old bug 0001231 with a similar issue, which in the history it >>> is pointed out that it may not be that new jobs can't spool while >>> existing jobs despool, but that the new jobs cannot verify that they >>> will have tape access, which is a step before spooling begins. >>> >>> I wonder if this is simply the state of affairs and if so, are there are >>> any plans to improve upon this 'inefficiency'. >>> >>> thanks! >>> Stephen > -- Stephen Thompson Berkeley Seismological Laboratory step...@seismo.berkeley.edu 215 McCone Hall # 4760 404.538.7077 (phone) University of California, Berkeley 510.643.5811 (fax) Berkeley, CA 94720-4760 ------------------------------------------------------------------------------ This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
