Re: amanda-2.4.4p3, file driver, chg-disk, and infinite driver retries
On Wed, Oct 20, 2004 at 10:52:47PM -0400, Todd Pfaff wrote: > > Jean-Louis supplied me with a patch which I modified slightly > and that I'm now using. He's good isn't he :)) > It turns out that some code in server-src/driver.c will allow a dump > to be retried indefinitely, if dumper returns TRYAGAIN and taper returns > DONE. See around line 2203 in 2.4.4p3 server-src/driver.c. > > ... . But consistently with vtapes, if a client is > alive during planning and was added to the schedule, and then this client > goes offline before the dump starts, this condition will occur whereby > dumper says TRYAGAIN, taper says DONE, forever until killed (or until > taper gets an error for some other reason I guess). Interesting edge case. Glad JLM was able to recognize it so quickly. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda-2.4.4p3, file driver, chg-disk, and infinite driver retries
On Wednesday 20 October 2004 22:33, Jon LaBadie wrote: >On Wed, Oct 20, 2004 at 10:08:59PM -0400, Gene Heskett wrote: >> On Wednesday 20 October 2004 17:54, Jon LaBadie wrote: >> >On Wed, Oct 20, 2004 at 05:02:38PM -0400, Todd Pfaff wrote: >> >> I'm using amanda-2.4.4p3 with the file driver and chg-disk >> >> changer for vtapes. >> >> >> >> Sometimes a dump fails and driver retries: >> >> >> >> driver: client /some/path 1 [dump to tape failed, will try >> >> again] >> >> >> >> and then it fails again and retries again, and this repeats >> >> until I kill the amanda processes. >> >> >> >> This last time this happened I caught it after >> >> driver/dumper/taper had created about 200 0 length dump files >> >> in the vtape directory. The client machine it was trying to >> >> dump from was offline. It had gone offline sometime after the >> >> planner stage and so the planner had included it. There was no >> >> indication that this retrying was ever going to stop. >> >> >> >> I have been using amanda for years with tapes and I never >> >> experienced this behaviour before so it came as quite a >> >> surprise. >> >> >> >> Is there any way to limit this retrying by driver? >> > >> >A guess only, might your vtape size be smaller than your DLE? >> >Similar to trying to write to a tape and reaching end of tape. >> >And what do you have "runtapes" set to. For real tapes that >> >specifies the maximum number of tapes that can be used for >> >any one amanda dump or flush. I guess that affects vtapes too. >> > >> >jl >> >> I think Jon, in that event (DLE too big for the tape) that it will >> simply go into an advance to the next vtape and retry till it runs >> out of vtapes (runtapes > 1 of course) to use loop. This happened >> to me when I was first setting my current lashup up. This walks >> and quacks like a different duck to me if it made 200 zero length >> files on the same vtape. > >I probably read more into it than was there. >I thought that each dump file was on a different tape. I didn't read that into it, but then the questioner could have worded it better, so its possible, but that would take 200 vtapes, and I don't think anyone has explored it that far for lack of sufficient storage space. And 200 50 megabyte vtapes on a 20GB drive would seem to be counter-productive even if it could be made to work with a disklist that almost defined each file. That would be like feeding the locate db into it for a disklist, very extreme. My 'common sense' if you can call it that, precluded drawing that conclusion. OTOH, my common sense has been known to be wrong too. :-) -- Cheers, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) 99.27% setiathome rank, not too shabby for a WV hillbilly Yahoo.com attorneys please note, additions to this message by Gene Heskett are: Copyright 2004 by Maurice Eugene Heskett, all rights reserved.
Re: amanda-2.4.4p3, file driver, chg-disk, and infinite driver retries
Thanks to Jon LaBadie, Gene Heskett, and Jean-Louis Martineau for replies. Jean-Louis supplied me with a patch which I modified slightly and that I'm now using. It turns out that some code in server-src/driver.c will allow a dump to be retried indefinitely, if dumper returns TRYAGAIN and taper returns DONE. See around line 2203 in 2.4.4p3 server-src/driver.c. I've been using amanda for years with real tapes but I've just started using vtapes this past week and this is the first time I've noticed such behaviour. This is a working amanda configuration that I've used to dump many hundreds of gigabytes already, both to real tapes in the past and to vtapes in recent days. But consistently with vtapes, if a client is alive during planning and was added to the schedule, and then this client goes offline before the dump starts, this condition will occur whereby dumper says TRYAGAIN, taper says DONE, forever until killed (or until taper gets an error for some other reason I guess). -- Todd Pfaff <[EMAIL PROTECTED]> Research & High-Performance Computing Support ABB 132, McMaster University, Hamilton, Ontario, Canada, L8S 4M1 phone: voice: 905-525-9140 ext 22920 fax: 905-528-3773
Re: amanda-2.4.4p3, file driver, chg-disk, and infinite driver retries
On Wed, Oct 20, 2004 at 10:08:59PM -0400, Gene Heskett wrote: > On Wednesday 20 October 2004 17:54, Jon LaBadie wrote: > >On Wed, Oct 20, 2004 at 05:02:38PM -0400, Todd Pfaff wrote: > >> I'm using amanda-2.4.4p3 with the file driver and chg-disk changer > >> for vtapes. > >> > >> Sometimes a dump fails and driver retries: > >> > >> driver: client /some/path 1 [dump to tape failed, will try > >> again] > >> > >> and then it fails again and retries again, and this repeats until > >> I kill the amanda processes. > >> > >> This last time this happened I caught it after driver/dumper/taper > >> had created about 200 0 length dump files in the vtape directory. > >> The client machine it was trying to dump from was offline. It had > >> gone offline sometime after the planner stage and so the planner > >> had included it. There was no indication that this retrying was > >> ever going to stop. > >> > >> I have been using amanda for years with tapes and I never > >> experienced this behaviour before so it came as quite a surprise. > >> > >> Is there any way to limit this retrying by driver? > > > >A guess only, might your vtape size be smaller than your DLE? > >Similar to trying to write to a tape and reaching end of tape. > >And what do you have "runtapes" set to. For real tapes that > >specifies the maximum number of tapes that can be used for > >any one amanda dump or flush. I guess that affects vtapes too. > > > >jl > > I think Jon, in that event (DLE too big for the tape) that it will > simply go into an advance to the next vtape and retry till it runs > out of vtapes (runtapes > 1 of course) to use loop. This happened to > me when I was first setting my current lashup up. This walks and > quacks like a different duck to me if it made 200 zero length files > on the same vtape. > I probably read more into it than was there. I thought that each dump file was on a different tape. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda-2.4.4p3, file driver, chg-disk, and infinite driver retries
On Wednesday 20 October 2004 17:54, Jon LaBadie wrote: >On Wed, Oct 20, 2004 at 05:02:38PM -0400, Todd Pfaff wrote: >> I'm using amanda-2.4.4p3 with the file driver and chg-disk changer >> for vtapes. >> >> Sometimes a dump fails and driver retries: >> >> driver: client /some/path 1 [dump to tape failed, will try >> again] >> >> and then it fails again and retries again, and this repeats until >> I kill the amanda processes. >> >> This last time this happened I caught it after driver/dumper/taper >> had created about 200 0 length dump files in the vtape directory. >> The client machine it was trying to dump from was offline. It had >> gone offline sometime after the planner stage and so the planner >> had included it. There was no indication that this retrying was >> ever going to stop. >> >> I have been using amanda for years with tapes and I never >> experienced this behaviour before so it came as quite a surprise. >> >> Is there any way to limit this retrying by driver? > >A guess only, might your vtape size be smaller than your DLE? >Similar to trying to write to a tape and reaching end of tape. >And what do you have "runtapes" set to. For real tapes that >specifies the maximum number of tapes that can be used for >any one amanda dump or flush. I guess that affects vtapes too. > >jl I think Jon, in that event (DLE too big for the tape) that it will simply go into an advance to the next vtape and retry till it runs out of vtapes (runtapes > 1 of course) to use loop. This happened to me when I was first setting my current lashup up. This walks and quacks like a different duck to me if it made 200 zero length files on the same vtape. -- Cheers, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) 99.27% setiathome rank, not too shabby for a WV hillbilly Yahoo.com attorneys please note, additions to this message by Gene Heskett are: Copyright 2004 by Maurice Eugene Heskett, all rights reserved.
Re: amanda-2.4.4p3, file driver, chg-disk, and infinite driver retries
On Wed, Oct 20, 2004 at 05:02:38PM -0400, Todd Pfaff wrote: > I'm using amanda-2.4.4p3 with the file driver and chg-disk changer for > vtapes. > > Sometimes a dump fails and driver retries: > > driver: client /some/path 1 [dump to tape failed, will try again] > > and then it fails again and retries again, and this repeats until I kill > the amanda processes. > > This last time this happened I caught it after driver/dumper/taper had > created about 200 0 length dump files in the vtape directory. The client > machine it was trying to dump from was offline. It had gone offline > sometime after the planner stage and so the planner had included it. > There was no indication that this retrying was ever going to stop. > > I have been using amanda for years with tapes and I never experienced > this behaviour before so it came as quite a surprise. > > Is there any way to limit this retrying by driver? A guess only, might your vtape size be smaller than your DLE? Similar to trying to write to a tape and reaching end of tape. And what do you have "runtapes" set to. For real tapes that specifies the maximum number of tapes that can be used for any one amanda dump or flush. I guess that affects vtapes too. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
