Re: [Bacula-users] [Non-DoD Source] Re: Bacula kills tape drive and autoloader
Hello Daniel, You are welcome :-). I see you are using /dev/sgX and /dev/nstX device names for the autochanger configuration. This is not recommended since they can be modified after server reboots. Please use /dev/tape/by-id values or even udev rules. Also, I noticed you have a pool configured to use "Storage = HP-Overland". To avoid confusion, I would rename your tape library and tape drive resources: > bacula-sd.conf > > > > Autochanger { > > Name = " > HP-Overland- > Autochanger" > > Device = HP-Overland > -Drive-0 > > Changer Command = "/etc/bacula/mtx-changer %c %o %S %a %d" > > Changer Device = /dev/tape/by-id/scsi-3500110a00093f5f0 > } > > > > Device { > > Name = HP-Overland > -Drive-0 > > Drive Index = 0 > > Media Type = LTO-5 > > Archive Device = /dev/tape/by-id/scsi-3500110a00093f5f1-nst > > AutomaticMount = yes; > > AlwaysOpen = yes; > > RemovableMedia = yes; > > RandomAccess = no; > > AutoChanger = yes; > > Alert Command = "sh -c 'smartctl -H -l error /dev/sg3'" > > } > The Storage definition in bacula-dir.conf, should use be something like: Storage { Name = HP-Overlan d -TL Device = HP-Overland-Autochanger ... Autochanger = yes } Then your Tape pool should be configured to use the tape library and not the single drive: # Tape pool definition > > Pool { > > Name = Tape > > Pool Type = Backup > > Recycle = yes > > AutoPrune = yes > > Volume Retention = 3 years > > Storage = HP-Overland-TL > } > Regarding what you mentioned about the migration job, I would recommend you to have the btape tests successfully run and a successful backup job writing to tape before dealing with a migration job. The "fill" btape test needs a blank tape. You should have it formated before using it with the fill btape test: mt -f /dev/nst0 rewind mt -f /dev/nst0 weof If this does not solve this problem, please let us know the Bacula version you are using. Best regards, Ana -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] A *different* append test problem with btape
Hello Steve, Great it worked! You are right, it is not a good idea to have both tape libraries share the same pool/media type. Mainly because LTO-6 is more than 2 generations ahead LTO-3. This means your LTO-6 drives will not be able to even read the LTO-3 tapes. I would recommend you to have a different pool for your new tape library and redirect the jobs to use the new one, if the old one will not be used for writing anymore. You may let it there until you have useful data to restore then. Best regards, Ana On Mon, Jul 24, 2017 at 4:14 PM, Steve Garciawrote: > > Thank you! > > This solved my problem. I was starting to worry that we had purchased a > fairly expensive drive that wasn't going to work -- tough to explain to my > boss. > > I had copied my configuration from our older instance of bacula, which is > using the same kind of library, but with the LTO3 version of the same model > tape drive. I'm not sure why this setting was defined in my older > configuration, but I set that up in 2009, so whatever reason I may have had > has completely faded. In any case, it's working with the old drive and > library, so I probably will leave it alone there. I've gotten rid of it > for the new library. > > Now I have to figure out a strategy for bringing the new library into the > mix. Other messages have let me know that my idea of making both part of > the same pool is not a good idea. > > "Ana Emília M. Arruda" wrote: > > Hello Steve, > > > > It seems this is due to "Offline On Unmount = yes". The default for this > > directive is no. Usually, it is not necessary to set this directive to > "no" > > when dealing with tape libraries. > > > > This will cause the tape to be offline after WEOF and not available to be > > used for append. > > > > Do you have any special reason to have this directive configured? > > > > Best regards, > > Ana > > > > El 20 jul. 2017 22:09, "Steve Garcia" escribió: > > > > > I have a *different* problem with a new tape drive and btape, also > failing > > > during the append test. > > > > > > The read/write test passes with no trouble, but the append test not > only > > > fails, but it takes the tape drive offline. Once this happens, the > drive > > > is not accessible until the tape is physically dismounted and then > > > remounted. Once this happens, the drive comes back to life, but it > will go > > > offline again if the append test is re-attempted. > > > > > > The btape failure message suggested using a fixed block size, but > making > > > that change didn't seem to make any difference. > > > > > > I tried downloading the IBM ITDT utility and running all its tests, but > > > the drive passed with flying colors in the ITDT diagnostics. > > > > > > I am running bacula 7.4.4 on a new Debian stretch (Debian 9) server > with a > > > new tape library and tape drive. The new drive is an IBM ULT3580-HH6 > LTO-6 > > > drive in a Dell TL-2000 library. I have an existing bacula instance > > > running on a Debian jesse (debian 8) server, and my plan is to add the > new > > > tape library as an additional pool for the existing instance. For now, > > > though, unless I can get the drive to work with bacula, planning how > to set > > > bacula itself up doesn't matter. > > > > > > What steps can I do to troubleshoot this? > > > > > > One thing I notice is that lsscsi shows the tape drive twice, but > there is > > > only one actual drive. Could this be a part of the problem? > > > > > > > > > root@odin:/etc/bacula/scripts# ./mtx-changer /dev/autochanger1 load 1 > > > /dev/nst0 0 > > > Loading media from Storage Element 1 into drive 0...done > > > > > > > > > root@odin:/etc/bacula/scripts# ./mtx-changer /dev/autochanger1 listall > > > D:0:F:1:15L6 > > > S:1:E > > > S:2:F:18L6 > > > S:3:F:21L6 > > > S:4:F:CLNU00L1 > > > S:5:F:14L6 > > > S:6:F:17L6 > > > S:7:F:20L6 > > > S:8:F:CLN005L3 > > > S:9:F:13L6 > > > S:10:F:16L6 > > > S:11:F:19L6 > > > S:12:F:12L6 > > > S:13:F:09L6 > > > S:14:F:06L6 > > > S:15:F:03L6 > > > S:16:F:11L6 > > > S:17:F:08L6 > > > S:18:F:05L6 > > > S:19:F:02L6 > > > S:20:F:10L6 > > > S:21:F:07L6 > > > S:22:F:04L6 > > > S:23:F:01L6 > > > I:24:E > > > > > > > > > root@odin:/etc/bacula/scripts# btape -c ../bacula-sd.conf /dev/nst0 > > > Tape block granularity is 1024 bytes. > > > btape: butil.c:291-0 Using device: "/dev/nst0" for writing. > > > test > > > btape: btape.c:471-0 open device "Drive-1" (/dev/nst0): OK > > > * > > > > > > === Write, rewind, and re-read test === > > > > > > I'm going to write 1 records and an EOF > > > then write 1 records and an EOF, then rewind, > > > and re-read the data to verify that it is correct. > > > > > > This is an *essential* feature ... > > > > > > btape: btape.c:1154-0 Wrote 1 blocks of 64412 bytes. > > > btape: btape.c:606-0 Wrote 1 EOF to "Drive-1" (/dev/nst0)
Re: [Bacula-users] A *different* append test problem with btape
Thank you! This solved my problem. I was starting to worry that we had purchased a fairly expensive drive that wasn't going to work -- tough to explain to my boss. I had copied my configuration from our older instance of bacula, which is using the same kind of library, but with the LTO3 version of the same model tape drive. I'm not sure why this setting was defined in my older configuration, but I set that up in 2009, so whatever reason I may have had has completely faded. In any case, it's working with the old drive and library, so I probably will leave it alone there. I've gotten rid of it for the new library. Now I have to figure out a strategy for bringing the new library into the mix. Other messages have let me know that my idea of making both part of the same pool is not a good idea. "Ana Emília M. Arruda"wrote: > Hello Steve, > > It seems this is due to "Offline On Unmount = yes". The default for this > directive is no. Usually, it is not necessary to set this directive to "no" > when dealing with tape libraries. > > This will cause the tape to be offline after WEOF and not available to be > used for append. > > Do you have any special reason to have this directive configured? > > Best regards, > Ana > > El 20 jul. 2017 22:09, "Steve Garcia" escribió: > > > I have a *different* problem with a new tape drive and btape, also failing > > during the append test. > > > > The read/write test passes with no trouble, but the append test not only > > fails, but it takes the tape drive offline. Once this happens, the drive > > is not accessible until the tape is physically dismounted and then > > remounted. Once this happens, the drive comes back to life, but it will go > > offline again if the append test is re-attempted. > > > > The btape failure message suggested using a fixed block size, but making > > that change didn't seem to make any difference. > > > > I tried downloading the IBM ITDT utility and running all its tests, but > > the drive passed with flying colors in the ITDT diagnostics. > > > > I am running bacula 7.4.4 on a new Debian stretch (Debian 9) server with a > > new tape library and tape drive. The new drive is an IBM ULT3580-HH6 LTO-6 > > drive in a Dell TL-2000 library. I have an existing bacula instance > > running on a Debian jesse (debian 8) server, and my plan is to add the new > > tape library as an additional pool for the existing instance. For now, > > though, unless I can get the drive to work with bacula, planning how to set > > bacula itself up doesn't matter. > > > > What steps can I do to troubleshoot this? > > > > One thing I notice is that lsscsi shows the tape drive twice, but there is > > only one actual drive. Could this be a part of the problem? > > > > > > root@odin:/etc/bacula/scripts# ./mtx-changer /dev/autochanger1 load 1 > > /dev/nst0 0 > > Loading media from Storage Element 1 into drive 0...done > > > > > > root@odin:/etc/bacula/scripts# ./mtx-changer /dev/autochanger1 listall > > D:0:F:1:15L6 > > S:1:E > > S:2:F:18L6 > > S:3:F:21L6 > > S:4:F:CLNU00L1 > > S:5:F:14L6 > > S:6:F:17L6 > > S:7:F:20L6 > > S:8:F:CLN005L3 > > S:9:F:13L6 > > S:10:F:16L6 > > S:11:F:19L6 > > S:12:F:12L6 > > S:13:F:09L6 > > S:14:F:06L6 > > S:15:F:03L6 > > S:16:F:11L6 > > S:17:F:08L6 > > S:18:F:05L6 > > S:19:F:02L6 > > S:20:F:10L6 > > S:21:F:07L6 > > S:22:F:04L6 > > S:23:F:01L6 > > I:24:E > > > > > > root@odin:/etc/bacula/scripts# btape -c ../bacula-sd.conf /dev/nst0 > > Tape block granularity is 1024 bytes. > > btape: butil.c:291-0 Using device: "/dev/nst0" for writing. > > test > > btape: btape.c:471-0 open device "Drive-1" (/dev/nst0): OK > > * > > > > === Write, rewind, and re-read test === > > > > I'm going to write 1 records and an EOF > > then write 1 records and an EOF, then rewind, > > and re-read the data to verify that it is correct. > > > > This is an *essential* feature ... > > > > btape: btape.c:1154-0 Wrote 1 blocks of 64412 bytes. > > btape: btape.c:606-0 Wrote 1 EOF to "Drive-1" (/dev/nst0) > > btape: btape.c:1170-0 Wrote 1 blocks of 64412 bytes. > > btape: btape.c:606-0 Wrote 1 EOF to "Drive-1" (/dev/nst0) > > btape: btape.c:1212-0 Rewind OK. > > 1 blocks re-read correctly. > > Got EOF on tape. > > 1 blocks re-read correctly. > > === Test Succeeded. End Write, rewind, and re-read test === > > > > btape: btape.c:1279-0 Block position test > > btape: btape.c:1291-0 Rewind OK. > > Reposition to file:block 0:4 > > Block 5 re-read correctly. > > Reposition to file:block 0:200 > > Block 201 re-read correctly. > > Reposition to file:block 0: > > Block 1 re-read correctly. > > Reposition to file:block 1:0 > > Block 10001 re-read correctly. > > Reposition to file:block 1:600 > > Block 10601 re-read correctly. > > Reposition to file:block 1: > > Block 2 re-read correctly. > > ===
Re: [Bacula-users] [Non-DoD Source] Re: Bacula kills tape drive and autoloader
Thanks for the reply Ana. Here is most of the info you asked. I will work on running btape again but now it is just timing out and saying that the device cannot be accessed when I try to run the fill test. [root@server ~]# lsscsi -g [0:2:0:0]diskLSI MR9260-8i2.13 /dev/sda /dev/sg0 [0:2:1:0]diskLSI MR9260-8i2.13 /dev/sdb /dev/sg1 [1:0:1:0]cd/dvd Slimtype DVD A DS8ACSH LP53 /dev/sr0 /dev/sg2 [5:0:4:0]tapeHP Ultrium 5-SCSI Z58W /dev/st0 /dev/sg3 [5:0:4:1]mediumx HP 1x8 G2 AUTOLDR 3.50 /dev/sch0 /dev/sg4 [root@server ~]# ls -laR /dev/tape /dev/tape: total 0 drwxr-xr-x 4 root root 80 Jul 21 11:43 . drwxr-xr-x 21 root root 3560 Jul 21 11:45 .. drwxr-xr-x 2 root root 100 Jul 21 11:45 by-id drwxr-xr-x 2 root root 80 Jul 21 11:43 by-path /dev/tape/by-id: total 0 drwxr-xr-x 2 root root 100 Jul 21 11:45 . drwxr-xr-x 4 root root 80 Jul 21 11:43 .. lrwxrwxrwx 1 root root 9 Jul 21 11:45 scsi-3500110a00093f5f0 -> ../../sg4 lrwxrwxrwx 1 root root 9 Jul 21 11:43 scsi-3500110a00093f5f1 -> ../../st0 lrwxrwxrwx 1 root root 10 Jul 21 11:43 scsi-3500110a00093f5f1-nst -> ../../nst0 /dev/tape/by-path: total 0 drwxr-xr-x 2 root root 80 Jul 21 11:43 . drwxr-xr-x 4 root root 80 Jul 21 11:43 .. lrwxrwxrwx 1 root root 9 Jul 21 11:43 pci-:04:00.0-sas-0x500110a00093f5f2-lun-0 -> ../../st0 lrwxrwxrwx 1 root root 10 Jul 21 11:43 pci-:04:00.0-sas-0x500110a00093f5f2-lun-0-nst -> ../../nst0 bacula-sd.conf Autochanger { Name = "Autochanger" Device = HP-Overland Changer Command = "/etc/bacula/mtx-changer %c %o %S %a %d" Changer Device = /dev/sg4 } Device { Name = HP-Overland Drive Index = 0 Media Type = LTO-5 Archive Device = /dev/nst0 AutomaticMount = yes; AlwaysOpen = yes; RemovableMedia = yes; RandomAccess = no; AutoChanger = yes; Alert Command = "sh -c 'smartctl -H -l error /dev/sg3'" } With the current configuration I am now able to complete the following, but when I run btape and try to run the fill test. The tape devices go offline. I need to power cycle the tape device manually. (stop bacula / storage daemon to run test) bacula stop or bacula-sd stop (load tape into slot 0 on autochanger) mtx -f /dev/sg4 load 1 TEST 1 - SUCCESSFUL mt -f /dev/nst0 rewind tar cvf /dev/nst0 . mt -f /dev/nst0 rewind tar tvf /dev/nst0 TEST 2 - SUCCESSFUL btape -c bacula-sd.conf /dev/nst0 * test (command within the btape console) TEST 3 - SUCCESSFUL btape -c bacula-sd.conf /dev/nst0 * autochanger (command within the btape console) TEST 4 - btape -c bacula-sd.conf /dev/nst0 * fill (command within the btape console) I am not sure if this is too much info. I have created a migration job. When I run it there are two jobs that are created, a migration and a backup but they just sit there waiting on tape device. bacula-dir.conf Job { Name = Migrate-to-Tape Type = Migrate Level = Full Client = server-fd FileSet = "Full Set" Schedule = SaturdayNight Pool = File Selection Type = SQLQuery Selection Pattern = " SELECT DISTINCT J.JobId, J.StartTime FROM Job J, Pool P WHERE P.Name = 'File' AND P.PoolId = J.PoolId AND J.Type = 'B' AND J.JobStatus IN ('T','W') AND J.JobBytes > 0 AND J.JobId NOT IN (SELECT PriorJobID FROM Job WHERE Type IN ('B','C') AND Job.JobStatus IN ('T','W') AND PriorJobId != 0) ORDER By J.StartTime LIMIT 1; " Purge Migration Job = yes Storage = HP-Overland Maximum Concurrent Jobs = 6 Messages = Standard Spool Data = yes Spool Attributes = yes } # Tape pool definition Pool { Name = Tape Pool Type = Backup Recycle = yes AutoPrune = yes Volume Retention = 3 years Storage = HP-Overland } # File Pool definition Pool { Name = File Pool Type = Backup Recycle = yes AutoPrune = yes Volume Retention = 30 days Maximum Volume Bytes = 500G Maximum Volumes = 100 Next Pool = Tape LabelFormat = FVol- } Daniel Hicks | 916-999-2711 | DMEA From: Ana Emília M. Arruda [mailto:emiliaarr...@gmail.com] Sent: Saturday, July 22, 2017 5:52 AM To: Hicks, Daniel CTR OSD DMEACc: Bacula-users@lists.sourceforge.net Subject: [Non-DoD Source] Re: [Bacula-users] Bacula kills tape drive and autoloader Hi Daniel, It would be helpful if you could send to the list the configuration you are using in Bacula for the Autochanger and the btape fill output. Also, it would be helpful the output of the following commands: lsscsi -g ls -laR /dev/tape Best regards, Ana On Fri, Jul 21, 2017 at 4:13 PM, Hicks, Daniel CTR OSD DMEA > wrote: Hello all I have been working on
[Bacula-users] Release 9.0.2
Hello, We are pleased to announce that we have just released Bacula version 9.0.2 This is a minor bug fix release, but a few of the bugs are important. The main items fixed are: – Postgresql should now work with Postgresql prior to 9.0 Note: the ssl connection feature added in 9.0 is not available on postgresql servers older than 9.0 (it needs the new connection API). – The issues with MariaDB (reconnect variable) are now fixed – The problem of the btape “test” command finding a wrong number of files in the append test was a bug. It is now fixed. It is unlikely that it affected anything but btape. – The bacula-tray-monitor.deskop script is released in the scripts directory. – We recommend that you build with both libz and lzo library support (the developer packages must be installed when building, and the shared object libraries must be installed at run time). However we have modified the code so that Bacula *should* build and run with either or both libz or lzo absent. 23Jul17 – Use Bacula in place of Libz variables so we can build with/without libz and lzo – Apply ideas from bug #2255 prettier status slots output – Configure and install bacula-tray-monitor.desktop – Fix btape test which counted files incorrectly on EOT – Fix bug #2296 where Bacula would not compile with postgres 8 or older – Fix bug #2294 Bacula does not build with MariaDB 10.2 – baculum: Fix multiple directors support – baculum: Fix showing errors from the API Bugs fixed/closed since last release: 2255 2294 2296 Thanks for using Bacula, Kern -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Baculum and LDAP authentication
Hello Marcin, thanks for your answer. I tried Baculum 7.4.7 with a plain configuration and auth user file "baculum.users" but got the same "Error 6". Baculum 9.0.1 is working on the same host without any problems. So it seems like the problem isn't related to LDAP based authentication. Regards, Daniel Am 21.07.17 um 13:28 schrieb Marcin Haba: Hello Daniel, I think that it isn't possible to use LDAP authentication with the Baculum 7.4. The Baculum 7.4 uses auth user file (baculum.users) to store users and this file is used both by web server and the Baculum itself. In the Baculum version 9.0 there are two baculum.users files: one for the web GUI and one for the API part. Also the meaning of the baculum.users file is changed in 9.0 because it is used only by web server, and the Baculum GUI and Baculum API only enable to save users to these files. I think that more possible it could be use LDAP authentication with the Baculum 9.0 (for Web GUI or API or both) but the important note is that in 9.0 the restricted consoles support is a bit changed. Restricted consoles are possible to define for OAuth2 per API client, but for Basic authentication the restricted consoles are unavailable. If you don't use restricted consoles then you can try with 9.0 version. Next week should be available binary packages for this version. Best regards. Marcin Haba (gani) On 21 July 2017 at 09:32, Daniel Heitepriemwrote: Hi everyone, I'm trying to get Baculum to work with LDAP authentication. The basic authentication when accessing the wizard of a fresh install is working fine but after completing the wizard I'm stuck as I get the error "Error 6 - authorization to Baculum error." and no error in Apache error log nor in baculum.dbg Below is my Apache Auth-config for Baculum and the content of baculum.dbg. Apache config: Order allow,deny Allow from all Options +FollowSymLinks RewriteEngine On RewriteCond %{REQUEST_FILENAME} !-d RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ index.php/$1 [L] AuthName "Baculum Auth" AuthBasicProvider ldap AuthType Basic AuthLDAPURL "ldap://IP:389/ou=staff,ou=people,dc=example,dc=com?uid; AuthLDAPBindDN "cn=baculum,ou=LDAPauth,dc=example,dc=com" AuthLDAPBindPassword "SecretPassword" Require valid-user LDAPReferrals Off baculum.dbg: Jul 21 07:31:52 [Debug] [Application] Command=setUrlParams, Output= EXECUTE URL ==> http://10.1.10.243:9095/directors?director= <== , File=/opt/baculum/protected/Class/API.php, Line=138 (line 89, /opt/baculum/protected/Class/Logging.php) Jul 21 07:31:53 [Debug] [Application] Command=preParseOutput, Output={"output":"Problem with authorization to Baculum WebGUI.","error":6}, File=/opt/baculum/protected/Class/API.php, Line=256 (line 89, /opt/baculum/protected/Class/Logging.php) Jul 21 07:31:53 [Debug] [Application] Command=preParseOutput, Output=stdClass Object ( [output] => Problem with authorization to Baculum WebGUI. [error] => 6 ) , File=/opt/baculum/protected/Class/API.php, Line=277 (line 89, /opt/baculum/protected/Class/Logging.php) Thanks in advance and regards, Daniel -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users