Re: [Bacula-users] Issue restoring from multiple tapes
Hello Alan, Yes your suggested change would be a reasonable feature to add. I cannot work on that right at the moment, so please submit a feature request. Best regards, Kern On 10/28/2017 11:39 PM, Alan Brown wrote: On 28/10/17 18:16, Kern Sibbald wrote: Hello Bill, By the way, I have just committed another patch for the problem of marking all volumes read-only. If you have time please test the latest code in the repo, I think it should correct the last note you put into your bug #2329. The case you cite below looks to me like Bacula is behaving as designed. Basically if the device is not there Bacula makes a few passes at trying to find it then simply fails the job. I am not sure how Bacula would "trap" this sort of a situation, and for me it really does not make sense for Bacula to notify the operator because it is not a simple mount request. Adding my 2 cents: It may not be a simple mount request, but the operator needs to know anyway, so it can be escalated quickly. I think the option of failing or hanging forever should be configurable. The operator will notice the problem when the job fails. Which is not ideal if it happens partway through a 2TB backup. Bottom line: if devices defined in the SD are not there, Bacula will after trying a few times fail the jobs. If you have a good idea on some other action, I am willing to listen. Best regards, Kern On 10/28/2017 06:54 PM, Bill Arlofski wrote: On 10/28/2017 10:10 AM, Phil Stracchino wrote: On 10/28/17 04:15, Kern Sibbald wrote: Hello, Thanks for the feedback. Can you confirm that your Bacula signs on with version 9.0.5? If so, it means that some recent patches that I have made for this problem (3-4 bug reports) solve the problem :-) I will definitely download and test. Hi Kern, Version 9.0.5 from git seems to mitigate this issue. (with a caveat or two). In my quick test here, I set up a restore job knowing full well that the storage array with the volumes required to do the restore was powered off - hoping to force Bacula to ask the operator for a volume and then wait. :) Bacula attempted to reserve, and then access each of the 6 disk devices in the autochanger. Of course it could not open any of them because the array was off/dismounted. It properly warned me for each device, then tried to loop through the 6 drives 3 more times (for a total of 4 loops), and then it marked the job as: "is waiting on Storage "aoe-file" So far so good. However, a few things: 1. It performed this loop every 30 seconds generating a lot of logging 2. It never mailed the operator to ask for a volume. 3. After pretty much exactly 10 minutes, the job was failed and a normal "job failure" email was sent to the admin. This second one might be the correct behavior since it is not trying to find a volume, it simply cannot access the defined drive devices -- something to think about... Should an operator be notified when a device cannot be opened by the SD? Attached is the first and last loop and job summary since it would just wrap horribly in this email. :) Kern, I understand that this test I just did may be a corner case and may just be throwing a monkey wrench into the mix, but similar scenarios have been seen in BEE Support, so it might be sensible to trap for this. Best regards, Bill -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
On 28/10/17 18:16, Kern Sibbald wrote: > > Hello Bill, > > By the way, I have just committed another patch for the problem of > marking all volumes read-only. If you have time please test the > latest code in the repo, I think it should correct the last note you > put into your bug #2329. > > The case you cite below looks to me like Bacula is behaving as > designed. Basically if the device is not there Bacula makes a few > passes at trying to find it then simply fails the job. I am not sure > how Bacula would "trap" this sort of a situation, and for me it really > does not make sense for Bacula to notify the operator because it is > not a simple mount request. > Adding my 2 cents: It may not be a simple mount request, but the operator needs to know anyway, so it can be escalated quickly. I think the option of failing or hanging forever should be configurable. > The operator will notice the problem when the job fails. > Which is not ideal if it happens partway through a 2TB backup. > Bottom line: if devices defined in the SD are not there, Bacula will > after trying a few times fail the jobs. > > If you have a good idea on some other action, I am willing to listen. > > Best regards, > > Kern > > > On 10/28/2017 06:54 PM, Bill Arlofski wrote: >> On 10/28/2017 10:10 AM, Phil Stracchino wrote: >>> On 10/28/17 04:15, Kern Sibbald wrote: Hello, Thanks for the feedback. Can you confirm that your Bacula signs on with version 9.0.5? If so, it means that some recent patches that I have made for this problem (3-4 bug reports) solve the problem :-) >>> I will definitely download and test. >> Hi Kern, >> >> Version 9.0.5 from git seems to mitigate this issue. (with a caveat or two). >> >> In my quick test here, I set up a restore job knowing full well that the >> storage array with the volumes required to do the restore was powered off - >> hoping to force Bacula to ask the operator for a volume and then wait. :) >> >> Bacula attempted to reserve, and then access each of the 6 disk devices in >> the >> autochanger. Of course it could not open any of them because the array was >> off/dismounted. It properly warned me for each device, then tried to loop >> through the 6 drives 3 more times (for a total of 4 loops), and then it >> marked >> the job as: >> >> "is waiting on Storage "aoe-file" >> >> So far so good. >> >> However, a few things: >> >> 1. It performed this loop every 30 seconds generating a lot of logging >> 2. It never mailed the operator to ask for a volume. >> 3. After pretty much exactly 10 minutes, the job was failed and a normal "job >> failure" email was sent to the admin. >> >> This second one might be the correct behavior since it is not trying to find >> a >> volume, it simply cannot access the defined drive devices -- something to >> think about... Should an operator be notified when a device cannot be opened >> by the SD? >> >> Attached is the first and last loop and job summary since it would just wrap >> horribly in this email. :) >> >> >> Kern, I understand that this test I just did may be a corner case and may >> just >> be throwing a monkey wrench into the mix, but similar scenarios have been >> seen >> in BEE Support, so it might be sensible to trap for this. >> >> Best regards, >> >> Bill >> >> >> >> >> >> -- >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >> >> >> ___ >> Bacula-users mailing list >> Bacula-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bacula-users > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
Yes, nice summary ("In other words") at the end :-)
On 10/28/2017 07:35 PM, Bill Arlofski wrote:
On 10/28/2017 11:16 AM, Kern Sibbald wrote:
Hello Bill,
By the way, I have just committed another patch for the problem of marking all
volumes read-only. If you have time please test the latest code in the repo,
I think it should correct the last note you put into your bug #2329.
OK! I will try to take a look now. Thanks for the heads-up.
The case you cite below looks to me like Bacula is behaving as designed.
Basically if the device is not there Bacula makes a few passes at trying to
find it then simply fails the job. I am not sure how Bacula would "trap" this
sort of a situation, and for me it really does not make sense for Bacula to
notify the operator because it is not a simple mount request. The operator
will notice the problem when the job fails. Bottom line: if devices defined
in the SD are not there, Bacula will after trying a few times fail the jobs.
If you have a good idea on some other action, I am willing to listen.
Yeah, I have nuthin' :)
As I think about this more, I am tending to agree with you here. :)
In other words:
- Devices configured for use should always be functioning and available, a job
will fail otherwise.
- Volumes may or may not be immediately available to the device and this is
perfectly normal; in which case an operator will be notified.
Best regards,
Bill
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
On 10/28/17 14:16, Phil Stracchino wrote: > On 10/28/17 13:41, Phil Stracchino wrote: >> On 10/28/17 12:10, Phil Stracchino wrote: >>> I will definitely download and test. >> >> Clean build on Solaris 11.3 amd64 and all three Gentoo Linux systems. > > Identified a tape-spanning file from my last full backup, and attempting > test restore now. Restore succeeded without incident and with no configuration changes. I believe bug #0002304 can be marked as fixed in 9.0.5. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
On 10/28/17 13:41, Phil Stracchino wrote: > On 10/28/17 12:10, Phil Stracchino wrote: >> I will definitely download and test. > > Clean build on Solaris 11.3 amd64 and all three Gentoo Linux systems. Identified a tape-spanning file from my last full backup, and attempting test restore now. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
On 10/28/17 12:10, Phil Stracchino wrote: > I will definitely download and test. Clean build on Solaris 11.3 amd64 and all three Gentoo Linux systems. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
On 10/28/2017 11:16 AM, Kern Sibbald wrote: > Hello Bill, > > By the way, I have just committed another patch for the problem of marking all > volumes read-only. If you have time please test the latest code in the repo, > I think it should correct the last note you put into your bug #2329. OK! I will try to take a look now. Thanks for the heads-up. > The case you cite below looks to me like Bacula is behaving as designed. > Basically if the device is not there Bacula makes a few passes at trying to > find it then simply fails the job. I am not sure how Bacula would "trap" this > sort of a situation, and for me it really does not make sense for Bacula to > notify the operator because it is not a simple mount request. The operator > will notice the problem when the job fails. Bottom line: if devices defined > in the SD are not there, Bacula will after trying a few times fail the jobs. > > If you have a good idea on some other action, I am willing to listen. Yeah, I have nuthin' :) As I think about this more, I am tending to agree with you here. :) In other words: - Devices configured for use should always be functioning and available, a job will fail otherwise. - Volumes may or may not be immediately available to the device and this is perfectly normal; in which case an operator will be notified. Best regards, Bill -- Bill Arlofski http://www.revpol.com/bacula -- Not responsible for anything below this line -- -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
Hello Bill, By the way, I have just committed another patch for the problem of marking all volumes read-only. If you have time please test the latest code in the repo, I think it should correct the last note you put into your bug #2329. The case you cite below looks to me like Bacula is behaving as designed. Basically if the device is not there Bacula makes a few passes at trying to find it then simply fails the job. I am not sure how Bacula would "trap" this sort of a situation, and for me it really does not make sense for Bacula to notify the operator because it is not a simple mount request. The operator will notice the problem when the job fails. Bottom line: if devices defined in the SD are not there, Bacula will after trying a few times fail the jobs. If you have a good idea on some other action, I am willing to listen. Best regards, Kern On 10/28/2017 06:54 PM, Bill Arlofski wrote: On 10/28/2017 10:10 AM, Phil Stracchino wrote: On 10/28/17 04:15, Kern Sibbald wrote: Hello, Thanks for the feedback. Can you confirm that your Bacula signs on with version 9.0.5? If so, it means that some recent patches that I have made for this problem (3-4 bug reports) solve the problem :-) I will definitely download and test. Hi Kern, Version 9.0.5 from git seems to mitigate this issue. (with a caveat or two). In my quick test here, I set up a restore job knowing full well that the storage array with the volumes required to do the restore was powered off - hoping to force Bacula to ask the operator for a volume and then wait. :) Bacula attempted to reserve, and then access each of the 6 disk devices in the autochanger. Of course it could not open any of them because the array was off/dismounted. It properly warned me for each device, then tried to loop through the 6 drives 3 more times (for a total of 4 loops), and then it marked the job as: "is waiting on Storage "aoe-file" So far so good. However, a few things: 1. It performed this loop every 30 seconds generating a lot of logging 2. It never mailed the operator to ask for a volume. 3. After pretty much exactly 10 minutes, the job was failed and a normal "job failure" email was sent to the admin. This second one might be the correct behavior since it is not trying to find a volume, it simply cannot access the defined drive devices -- something to think about... Should an operator be notified when a device cannot be opened by the SD? Attached is the first and last loop and job summary since it would just wrap horribly in this email. :) Kern, I understand that this test I just did may be a corner case and may just be throwing a monkey wrench into the mix, but similar scenarios have been seen in BEE Support, so it might be sensible to trap for this. Best regards, Bill -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
On 10/28/2017 10:10 AM, Phil Stracchino wrote: > On 10/28/17 04:15, Kern Sibbald wrote: >> Hello, >> >> Thanks for the feedback. Can you confirm that your Bacula signs on with >> version 9.0.5? If so, it means that some recent patches that I have >> made for this problem (3-4 bug reports) solve the problem :-) > > I will definitely download and test. Hi Kern, Version 9.0.5 from git seems to mitigate this issue. (with a caveat or two). In my quick test here, I set up a restore job knowing full well that the storage array with the volumes required to do the restore was powered off - hoping to force Bacula to ask the operator for a volume and then wait. :) Bacula attempted to reserve, and then access each of the 6 disk devices in the autochanger. Of course it could not open any of them because the array was off/dismounted. It properly warned me for each device, then tried to loop through the 6 drives 3 more times (for a total of 4 loops), and then it marked the job as: "is waiting on Storage "aoe-file" So far so good. However, a few things: 1. It performed this loop every 30 seconds generating a lot of logging 2. It never mailed the operator to ask for a volume. 3. After pretty much exactly 10 minutes, the job was failed and a normal "job failure" email was sent to the admin. This second one might be the correct behavior since it is not trying to find a volume, it simply cannot access the defined drive devices -- something to think about... Should an operator be notified when a device cannot be opened by the SD? Attached is the first and last loop and job summary since it would just wrap horribly in this email. :) Kern, I understand that this test I just did may be a corner case and may just be throwing a monkey wrench into the mix, but similar scenarios have been seen in BEE Support, so it might be sensible to trap for this. Best regards, Bill -- Bill Arlofski http://www.revpol.com/bacula -- Not responsible for anything below this line -- *llist joblog jobid=17968 time: 2017-10-28 10:27:30 logtext: bacula-dir JobId 17968: Start Restore Job Restore.2017-10-28_10.27.28_53 time: 2017-10-28 10:27:30 logtext: bacula-sd JobId 17968: Error: init_dev.c:123 [SE0001] Unable to stat device AoE-Changer0_drive0 at /mnt/Coraid/AoE-File0: ERR=Permission denied time: 2017-10-28 10:27:30 logtext: bacula-sd JobId 17968: Warning: Device "AoE-Changer0_drive0" in changer "AoE-Changer0" requested by DIR could not be opened or does not exist. time: 2017-10-28 10:27:30 logtext: bacula-sd JobId 17968: Error: init_dev.c:123 [SE0001] Unable to stat device AoE-Changer0_drive1 at /mnt/Coraid/AoE-File0: ERR=Permission denied time: 2017-10-28 10:27:30 logtext: bacula-sd JobId 17968: Warning: Device "AoE-Changer0_drive1" in changer "AoE-Changer0" requested by DIR could not be opened or does not exist. time: 2017-10-28 10:27:30 logtext: bacula-sd JobId 17968: Error: init_dev.c:123 [SE0001] Unable to stat device AoE-Changer0_drive2 at /mnt/Coraid/AoE-File0: ERR=Permission denied time: 2017-10-28 10:27:30 logtext: bacula-sd JobId 17968: Warning: Device "AoE-Changer0_drive2" in changer "AoE-Changer0" requested by DIR could not be opened or does not exist. time: 2017-10-28 10:27:30 logtext: bacula-sd JobId 17968: Error: init_dev.c:123 [SE0001] Unable to stat device AoE-Changer0_drive3 at /mnt/Coraid/AoE-File0: ERR=Permission denied time: 2017-10-28 10:27:30 logtext: bacula-sd JobId 17968: Warning: Device "AoE-Changer0_drive3" in changer "AoE-Changer0" requested by DIR could not be opened or does not exist. time: 2017-10-28 10:27:30 logtext: bacula-sd JobId 17968: Error: init_dev.c:123 [SE0001] Unable to stat device AoE-Changer0_drive4 at /mnt/Coraid/AoE-File0: ERR=Permission denied time: 2017-10-28 10:27:30 logtext: bacula-sd JobId 17968: Warning: Device "AoE-Changer0_drive4" in changer "AoE-Changer0" requested by DIR could not be opened or does not exist. time: 2017-10-28 10:27:30 logtext: bacula-sd JobId 17968: Error: init_dev.c:123 [SE0001] Unable to stat device AoE-Changer0_drive5 at /mnt/Coraid/AoE-File0: ERR=Permission denied time: 2017-10-28 10:27:30 logtext: bacula-sd JobId 17968: Warning: Device "AoE-Changer0_drive5" in changer "AoE-Changer0" requested by DIR could not be opened or does not exist. (about 3600 more log entries snipped) (last loop and job summary) time: 2017-10-28 10:37:30 logtext: bacula-sd JobId 17968: Error: init_dev.c:123 [SE0001] Unable to stat device AoE-Changer0_drive0 at /mnt/Coraid/AoE-File0: ERR=Permission denied time: 2017-10-28 10:37:30 logtext: bacula-sd JobId 17968: Warning: Device "AoE-Changer0_drive0" in changer "AoE-Changer0" requested by DIR could not be opened or does not exist. time: 2017-10-28 10:37:30 logtext: bacula-sd JobId 17968: Error: init_dev.c:123 [SE0001] Unable to stat device AoE-Changer0_drive1 at
Re: [Bacula-users] Issue restoring from multiple tapes
On 10/28/17 04:15, Kern Sibbald wrote: > Hello, > > Thanks for the feedback. Can you confirm that your Bacula signs on with > version 9.0.5? If so, it means that some recent patches that I have > made for this problem (3-4 bug reports) solve the problem :-) I will definitely download and test. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
Yes bconsole reports it is version 9.0.5 dated Oct 23 2017. I left a restore and made it wait a few hours between tape changes and it patiently waited and continued on without issue so far. On 10/28/2017 04:15 AM, Kern Sibbald wrote: > Hello, > > Thanks for the feedback. Can you confirm that your Bacula signs on with > version 9.0.5? If so, it means that some recent patches that I have > made for this problem (3-4 bug reports) solve the problem :-) > > Best regards, > > Kern > > PS: I have many other backports and other work to do to Bacula in the > near future, but once I get a confirmation on this particular problem > (huge number of messages in certain situations when the SD should wait), > I will officially release a new version. > > > > On 10/28/2017 06:22 AM, Andrew Ryder wrote: >> Try pulling down the latest git version of bacula 9.0.x .. I pulled it >> down and so far it politely asked for the next tape and patiently waited >> for it to be loaded. >> >> On 09/25/2017 12:59 AM, Phil Stracchino wrote: >>> On 09/24/17 20:24, Andrew Ryder wrote: Hello, I'm having an issue after upgrading from version 7 to version 9 where if I'm restoring from a job that spans more than one tape the job fails as soon as bacula hits the end of the first tape. Is there a new configuration option or something that needs to be set so it will wait for me to load the next tape rather than failing? >>> >>> I have encountered this same problem too. I have yet to find a solution >>> to it except to resume from the first file on the next tape, and try to >>> find the file that crosses the tape ends from a different backup. I >>> have filed a bug report on it. The basic problem seems to be that >>> Bacula 9 is polling the drive many, many times a second after the first >>> tape ends, but gives up after ten failed polls, which isn't remotely >>> enough time to even unload the tape let alone change it. >>> >>> >> -- >> >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >> ___ >> Bacula-users mailing list >> Bacula-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bacula-users >> > > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
Hi Andrew, hi Phil,
I have had a similar problem with making a backup spanning mutiple drives (no
autochanger, so I need to change the tapes by hand). I Have set "some" options
to circumvent it. I didn't do to much research, which one was the critical one,
but maybe that might help you out:
my configuration for the LTO-storage looks like this:
Device {
Name = LTO-5
Archive Device = /dev/nst0
Device Type = Tape
Media Type = LTO-5
LabelMedia = yes
Random Access = no
AutomaticMount = yes
RemovableMedia = yes
AlwaysOpen = yes
Maximum Concurrent Jobs = 1
LabelMedia = yes
Maximum File Size = 400G
Maximum Changer Wait = 2h
Volume Poll Interval = 0
}
In the job-definitions (bacula-dir.conf), I added the option "Max Wait Time":
Job {
Name = "XYZ LTO-5"
Client = XXX
JobDefs = DefaultJob
FileSet = "ZZZ"
Write Bootstrap = "/var/lib/bacula/XXX-ZZZ.bsr"
Priority = 31
Storage = LTO-5HH
Pool = LTO-5HH-Pool
Schedule = "WeeklyCycle"
Max Wait Time = 5d
}
I hope this does the trick for you, too.
Best,
Christian
-----Original-Nachricht-----
Betreff: Re: [Bacula-users] Issue restoring from multiple tapes
Datum: 2017-10-28T06:15:05+0200
Von: "Andrew Ryder" <tire...@shaw.ca>
An: "Phil Stracchino" <ph...@caerllewys.net>,
"bacula-users@lists.sourceforge.net" <bacula-users@lists.sourceforge.net>
Try pulling down the latest git version of bacula 9.0.x .. I pulled it
down and so far it politely asked for the next tape and patiently waited
for it to be loaded.
On 09/25/2017 12:59 AM, Phil Stracchino wrote:
> On 09/24/17 20:24, Andrew Ryder wrote:
>> Hello,
>>
>> I'm having an issue after upgrading from version 7 to version 9 where if
>> I'm restoring from a job that spans more than one tape the job fails as
>> soon as bacula hits the end of the first tape. Is there a new
>> configuration option or something that needs to be set so it will wait
>> for me to load the next tape rather than failing?
>
>
> I have encountered this same problem too. I have yet to find a solution
> to it except to resume from the first file on the next tape, and try to
> find the file that crosses the tape ends from a different backup. I
> have filed a bug report on it. The basic problem seems to be that
> Bacula 9 is polling the drive many, many times a second after the first
> tape ends, but gives up after ten failed polls, which isn't remotely
> enough time to even unload the tape let alone change it.
>
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
Hello, Thanks for the feedback. Can you confirm that your Bacula signs on with version 9.0.5? If so, it means that some recent patches that I have made for this problem (3-4 bug reports) solve the problem :-) Best regards, Kern PS: I have many other backports and other work to do to Bacula in the near future, but once I get a confirmation on this particular problem (huge number of messages in certain situations when the SD should wait), I will officially release a new version. On 10/28/2017 06:22 AM, Andrew Ryder wrote: Try pulling down the latest git version of bacula 9.0.x .. I pulled it down and so far it politely asked for the next tape and patiently waited for it to be loaded. On 09/25/2017 12:59 AM, Phil Stracchino wrote: On 09/24/17 20:24, Andrew Ryder wrote: Hello, I'm having an issue after upgrading from version 7 to version 9 where if I'm restoring from a job that spans more than one tape the job fails as soon as bacula hits the end of the first tape. Is there a new configuration option or something that needs to be set so it will wait for me to load the next tape rather than failing? I have encountered this same problem too. I have yet to find a solution to it except to resume from the first file on the next tape, and try to find the file that crosses the tape ends from a different backup. I have filed a bug report on it. The basic problem seems to be that Bacula 9 is polling the drive many, many times a second after the first tape ends, but gives up after ten failed polls, which isn't remotely enough time to even unload the tape let alone change it. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
Try pulling down the latest git version of bacula 9.0.x .. I pulled it down and so far it politely asked for the next tape and patiently waited for it to be loaded. On 09/25/2017 12:59 AM, Phil Stracchino wrote: > On 09/24/17 20:24, Andrew Ryder wrote: >> Hello, >> >> I'm having an issue after upgrading from version 7 to version 9 where if >> I'm restoring from a job that spans more than one tape the job fails as >> soon as bacula hits the end of the first tape. Is there a new >> configuration option or something that needs to be set so it will wait >> for me to load the next tape rather than failing? > > > I have encountered this same problem too. I have yet to find a solution > to it except to resume from the first file on the next tape, and try to > find the file that crosses the tape ends from a different backup. I > have filed a bug report on it. The basic problem seems to be that > Bacula 9 is polling the drive many, many times a second after the first > tape ends, but gives up after ten failed polls, which isn't remotely > enough time to even unload the tape let alone change it. > > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
Am 05.10.2017 um 16:11 schrieb Phil Stracchino: > I didn't realize that you were using an autochanger > rather than a single drive. Last week I wrote about my intention for an autochanger test - after going to the Bacula 9.0.x. versions recently. -- Ralf Brnkmann -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
Am 04.10.2017 um 15:44 schrieb Phil Stracchino: On 10/04/17 04:13, Ralf Brinkmann wrote: Thanks Phil, I found two files in two different jobs that both span the same two LTO-6 cassettes. The two restore jobs for a single file each did require two cassettes and did work without any error message. A compare with md5sum found the restored files are o.k. Interesting in several ways. How did you run a two-tape restore without the restore timing out at tape change? Just took a look into the logfiles of the restore test with our LTO6 HP 1/8 G2 autochanger - the point of the cassette change: 04-Okt 09:22 lx-vihost-1-sd JobId 2917: End of Volume "FR0401L6" at addr=0:0 on device "tape1" (/dev/nst0). 04-Okt 09:22 lx-vihost-1-sd JobId 2917: 3307 Issuing autochanger "unload Volume FR0401L6, Slot 5, Drive 0" command. 04-Okt 09:23 lx-vihost-1-sd JobId 2917: 3304 Issuing autochanger "load Volume FR0402L6, Slot 6, Drive 0" command. 04-Okt 09:24 lx-vihost-1-sd JobId 2917: 3305 Autochanger "load Volume FR0402L6, Slot 6, Drive 0", status is OK. 04-Okt 09:24 lx-vihost-1-sd JobId 2917: Ready to read from volume "FR0402L6" on Tape device "tape1" (/dev/nst0). 04-Okt 09:24 lx-vihost-1-sd JobId 2917: Forward spacing Volume "FR0402L6" to addr=0:1 -- Ralf Brinkmann -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
On 09/29/17 13:54, Phil Stracchino wrote: > On 09/29/17 02:14, Ralf Brinkmann wrote: >> Has someone a Mysql query for on an autochanger test to pickup files >> that crosses tape boundary? > > HThat's a good question. Let me think about that one for a > few minutes. There isn't a practical way to do it in a single query, but you could probably write a stored procedure to do it. First we need to find out what Volumes were used for the Job and in which order: MySQL localhost> select distinct Media.MediaId, Media.VolumeName from Media join JobMedia on JobMedia.MediaId = Media.MediaId where JobMedia.JobId = 15014 order by Media.LastWritten; +-++ | MediaId | VolumeName | +-++ | 48 | LTO4-FULL-0013 | |2426 | LTO4-FULL-0014 | | 51 | LTO4-FULL-0016 | +-++ 3 rows in set (0.00 sec) Now we need to know what the first and last file indexes on each of those volumes was. We need three separate queries to do this, one for each volume, because we can't call min() and max() on values corresponding to multiple keys at once. If we do that, we'll get a single pair of results containing the first and last indexes in the entire job, which isn't what we want. We need to go one MediaId at a time: MySQL localhost> select MediaId, min(FirstIndex), max(LastIndex) from JobMedia where JobId = 15014 and MediaId = 48; +-+-++ | MediaId | min(FirstIndex) | max(LastIndex) | +-+-++ | 48 | 1 | 145851 | +-+-++ 1 row in set (0.00 sec) MySQL localhost> select MediaId, min(FirstIndex), max(LastIndex) from JobMedia where JobId = 15014 and MediaId = 2426; +-+-++ | MediaId | min(FirstIndex) | max(LastIndex) | +-+-++ |2426 | 145851 | 146663 | +-+-++ 1 row in set (0.00 sec) MySQL localhost> select MediaId, min(FirstIndex), max(LastIndex) from JobMedia where JobId = 15014 and MediaId = 51; +-+-++ | MediaId | min(FirstIndex) | max(LastIndex) | +-+-++ | 51 | 146663 | 150149 | +-+-++ 1 row in set (0.00 sec) Uh-oh. Volume 48's last file has the same FileIndex as volume 2426's first file, and volume 2426's last file has the same FileIndex as volume 51's first file. Houston, we have a problem. Let's find out which files these are. To do that, we'll need to look them up by their FileIndexes and the JobId: MySQL localhost> select FileIndex, Path, Name from File join Filename on File.FilenameId = Filename.FilenameId join Path on File.PathId = Path.PathId where JobId = 15014 and (FileIndex = 145851 or FileIndex = 146663); +---++---+ | FileIndex | Path | Name | +---++---+ |145851 | /netstore/scratch/NIN - The Slip - 96-24 High Res/ | 10 Demon Seed.wav | |146663 | /netstore/video/TV/Criminal Minds/Criminal Minds Season 3/ | Criminal Minds S03E17 In Heat.avi | +---++---+ 2 rows in set (0.31 sec) And there you go. Those two files span tape boundaries. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
On 09/29/17 02:14, Ralf Brinkmann wrote: > Has someone a Mysql query for on an autochanger test to pickup files > that crosses tape boundary? HThat's a good question. Let me think about that one for a few minutes. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
On 09/24/17 20:24, Andrew Ryder wrote: > Hello, > > I'm having an issue after upgrading from version 7 to version 9 where if > I'm restoring from a job that spans more than one tape the job fails as > soon as bacula hits the end of the first tape. Is there a new > configuration option or something that needs to be set so it will wait > for me to load the next tape rather than failing? This bug was reported on August 10 as bug #0002304. It is possible to mostly work around it by resuming from the first file on the next tape as a separate backup job. The file that spans the tape change will have to be restored from a different backup in which it does not span a tape boundary. Any single file larger than a single tape is probably not restorable until bug #0002304 is fixed. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
Thanks. I'll have to keep a lookout whenever the next update comes out to see if it gets patched then. On 09/28/2017 09:31 AM, Phil Stracchino wrote: On 09/24/17 20:24, Andrew Ryder wrote: Hello, I'm having an issue after upgrading from version 7 to version 9 where if I'm restoring from a job that spans more than one tape the job fails as soon as bacula hits the end of the first tape. Is there a new configuration option or something that needs to be set so it will wait for me to load the next tape rather than failing? This bug was reported on August 10 as bug #0002304. It is possible to mostly work around it by resuming from the first file on the next tape as a separate backup job. The file that spans the tape change will have to be restored from a different backup in which it does not span a tape boundary. Any single file larger than a single tape is probably not restorable until bug #0002304 is fixed. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issue restoring from multiple tapes
On 09/24/17 20:24, Andrew Ryder wrote: > Hello, > > I'm having an issue after upgrading from version 7 to version 9 where if > I'm restoring from a job that spans more than one tape the job fails as > soon as bacula hits the end of the first tape. Is there a new > configuration option or something that needs to be set so it will wait > for me to load the next tape rather than failing? I have encountered this same problem too. I have yet to find a solution to it except to resume from the first file on the next tape, and try to find the file that crosses the tape ends from a different backup. I have filed a bug report on it. The basic problem seems to be that Bacula 9 is polling the drive many, many times a second after the first tape ends, but gives up after ten failed polls, which isn't remotely enough time to even unload the tape let alone change it. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
