Re: [Bacula-users] Looking for a LTO autoloader (NeoT24 works)
Sorry it’s late – I don’t read the Bacula list every day. We successfully use a Neo T24, LTO-6, SAS interface, LSI SAS 9207-8e controller, on UB 18.04LTS in ESXi 6.0 (you have to use hardware passthrough on the SAS controller card to use a SAS tape drive in ESXi). We previously used a Tandberg Magnum 224 built circa 2009. Except for SCSI interface & LTO-4 speed/capacity, it was a better unit – more steel, less plastic, easier to change tapes, better front-panel interface. That’s the limit of my experience with autoloaders, so I can’t offer a brand-to-brand comparison. Good luck, Dan Stieneke IT Specialist USDA - ARS - NWISRL 3793 N 3600 E Kimberly, ID 83341 208/423-6519 From: Markus Falb Sent: Monday, November 12, 2018 4:35 PM To: bacula-users@lists.sourceforge.net Subject: [Bacula-users] Looking for a LTO autoloader Hello, I am considering a purchase of a relatively small autoloader, but am not sure about compatibility with bacula. I found this one: https://www.overlandstorage.com/products/tape-libraries-and-autoloaders/neos-t24.aspx#Overview If anyone on this list has experiences with this autoloader + bacula, please let me know. I does know there is a "supported" list in the documentation http://www.bacula.org/9.2.x-manuals/en/main/Supported_Autochangers.html but don't know how comprehensive it is, there are some Overland models listed, but not this specific model (There doesn't exist a "not supported" list, does it? ) Also, suggestions (or dis-recommendations) for other autoloaders are appreciated, of course. Best Regards, Markus P.S. the Qualstar Q24 looks pretty identical to the NEOs T24 https://www.qualstar.com/qsr-q24.php Does anyone know what the differences are? This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately. ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] MANY duplicate "Max Volume jobs=X exceeded" messages in log
My reason for setting "Max Volume jobs=1" is that migrations will not occur
unless a tape is marked full, used, or error. To shorten the backup window, I
back up to striped SSDs, and from there migrate onto tape. Setting max jobs to
1 ensures that last disk volume is marked "used", rather than "append". I
couldn't think of an easier way to do this than setting max jobs to 1; what is
the preferred way to do it?
My pool definition is:
Pool {
Name = tempDisk-pool
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Retention = 5 days # We want to recycle this thing next
week
Maximum Volume Bytes = 100G
Maximum Volumes = 20# not all of them have to be full
sized??
Maximum Volume Jobs = 1 # migrations will not occur unless
volume marked full,used,error.
# append won't migrate to prevent
attempts to read & write simultaneously
# last volume in set will be 'append',
unless it's only allowed one job
Volume Use Duration = 23 hours
Recycle Oldest Volume = yes # This one respects retention times
Label Format = "tempDisk-"
Action on Purge = Truncate
Storage = File1
Next Pool = weeklyMain-LTO5-pool
}
Thanks for looking at this,
Dan Stieneke
IT Specialist
USDA - ARS - NWISRL
3793 N 3600 E
Kimberly, ID 83341
208/423-6519
-Original Message-
From: Phil Stracchino
Sent: Monday, October 1, 2018 2:45 PM
To: bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] MANY duplicate "Max Volume jobs=X exceeded"
messages in log
On 10/1/18 4:32 PM, Stieneke, Dan wrote:
> 64-bit Ubuntu 18.04LTS using distro-packaged Bacula, v9.0.6.
>
>
> Backing up to 100G disk volumes, max job=1, I get 107 lines in the
> output for each volume:
>
>
> 01-Oct 12:47 -dir JobId 23: Max Volume jobs=1 exceeded.
> Marking Volume "-0056" as Used.
>
>
> The 100G disk volumes are **107**,374,158,233 bytes long, so every ~1GB
> the message appears. I'd much rather have only one warning than 107.
>
> Is there a setting I'm missing somewhere?
Please include your Pool definition. It looks as though you have
misunderstood and misconfigured something, but without seeing your
configuration we can only make guesses as to what.
That said, "Max Volume jobs=1" is USUALLY a bad idea unless you have
specific reasons for doing it that way.
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
This electronic message contains information generated by the USDA solely for
the intended recipients. Any unauthorized interception of this message or the
use or disclosure of the information it contains may violate the law and
subject the violator to civil or criminal penalties. If you believe you have
received this message in error, please notify the sender and delete the email
immediately.
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] MANY duplicate "Max Volume jobs=X exceeded" messages in log
64-bit Ubuntu 18.04LTS using distro-packaged Bacula, v9.0.6. Backing up to 100G disk volumes, max job=1, I get 107 lines in the output for each volume: 01-Oct 12:47 -dir JobId 23: Max Volume jobs=1 exceeded. Marking Volume "-0056" as Used. The 100G disk volumes are *107*,374,158,233 bytes long, so every ~1GB the message appears. I'd much rather have only one warning than 107. Is there a setting I'm missing somewhere? Thanks, Dan Stieneke IT Specialist USDA - ARS - NWISRL 3793 N 3600 E Kimberly, ID 83341 208/423-6519 This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately. ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula h/w write fails, but tar writes w/out error?
Thanks all for your input & confirming it pretty much had to be a hardware
problem.
In the interest of completeness / helping the next person who's googling for
answers, reseating the SCSI card fixed it - it just completed a 900GB backup
w/out any problems, onto one of the same tapes that it had rejected before
after only a few GB.
Now that the major problem has been solved, I'm still curious about why Bacula
ran into the (real!) hardware issue where tar did not. The tar tape was
software compressed & then software encrypted, so the restore had to
successfully decrypt & then decompress the data, so there couldn't have been
any bit errors on that tar tape. This was true four months ago, with the
sketchy cable, and this time, with the SCSI card that needed re-seated. Are
fixed-size (tar) blocks just a little bit more robust than variable-sized
(Bacula) blocks?
And thanks, Kern, for an outstanding product.
Dan Stieneke
IT Specialist
USDA - ARS - NWISRL
3793 N 3600 E
Kimberly, ID 83341
208/423-6519
-Original Message-
From: Kern Sibbald [mailto:k...@sibbald.com]
Sent: Saturday, June 9, 2018 4:16 AM
To: Stieneke, Dan
Cc: bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] Bacula h/w write fails, but tar writes w/out error?
Hello,
Well, Bacula does not check what was written from time to time, but when it
reaches the end of the tape, Bacula will re-read the last block written to make
sure it corresponds to what it wrote, then it writes a double end of file. In
your case, something is going wrong -- either there is a hardware error, or
there is really an end of tape marker that is telling Bacula that the tape is
full. From what you write, it looks more like a hardware error, and the kernel
logs that you show below indicate that something serious is wrong with your
tape drive. While Bacula is writing you should never see such messages, and
when they occur, Bacula will receive a write error. Everything is consistent
with a hardware problem. You may get a better idea of what is going on by
running the "btape test" command. Please see the manual for instructions on
how to run it. I recommend both the test, and the fill commands. Note: both
of these commands will write on the tape. Prior to using a tape with btape, if
it has been labeled by Bacula, you should rewind the tape and write one or two
eof marks at the beginning so that btape will take it as a blank tape.
If both btape "test" and "fill" work, you should not have problems with failing
Bacula backups. If either one of those tests fail, you must fix it prior to
trying to backup on tape with Bacula.
Best regards,
Kern
On 06/08/2018 07:48 PM, Stieneke, Dan wrote:
> @ Dan Langille - yes, I think it is an issue with the tape drive, but only
> Bacula runs into it; tar does not.
>
> @Martin Simmons - of course I should have checked/reported the log, sorry.
> ===BEGIN SYSLOG
> ==
> Jun 4 08:06:11 SRVName kernel: [410468.465702] st0: Sense
> Key : Unit Attention [current] Jun 4 08:06:11 SRVName kernel:
> [410468.465714] st0: Add. Sense: Power on, reset, or bus device reset
> occurred Jun 4 08:10:47 SRVName kernel: [410744.629015] st0: Sense
> Key : Unit Attention [current] Jun 4 08:10:47 SRVName kernel:
> [410744.629026] st0: Add. Sense: Power on, reset, or bus device reset
> occurred Jun 4 08:14:02 SRVName kernel: [410939.819168] st0: Sense
> Key : Unit Attention [current] Jun 4 08:14:02 SRVName kernel:
> [410939.819180] st0: Add. Sense: Power on, reset, or bus device reset
> occurred Jun 4 08:16:57 SRVName kernel: [44.538975] st0: Sense
> Key : Unit Attention [current] Jun 4 08:16:57 SRVName kernel:
> [44.538988] st0: Add. Sense: Power on, reset, or bus device reset
> occurred ===END SYSLOG
> ==
>
>
> Googling for those entries I found
> http://bacula.10910.n7.nabble.com/Bacula-tapes-marked-FULL-too-early-VolBytes-too-low-td58881i20.html.
> Similar issue (but no report of tar), the thread ended with "similar problem
> went away with replaced drive" & "get your drive tested"
>
> From the Bacula log ("Error: Re-read of last block OK, but block numbers
> differ. Read block=990557 Want block=990558.") it looks like Bacula checks up
> on what has been written every so often. I don't think tar does that; it just
> streams to tape. If my card/cable/tape is only slightly flaky, is it
> reasonable to think that this extra work pushes it over the edge? Or am I
> barking up the wrong tree?
>
> Thanks,
> Dan Stieneke
>
>
> - from Dan Langille -
> If it is all tapes, is the issue with the tape drive?
>
&
Re: [Bacula-users] Bacula h/w write fails, but tar writes w/out error?
@ Dan Langille - yes, I think it is an issue with the tape drive, but only
Bacula runs into it; tar does not.
@Martin Simmons - of course I should have checked/reported the log, sorry.
===BEGIN SYSLOG
==
Jun 4 08:06:11 SRVName kernel: [410468.465702] st0: Sense Key : Unit Attention
[current]
Jun 4 08:06:11 SRVName kernel: [410468.465714] st0: Add. Sense: Power on,
reset, or bus device reset occurred
Jun 4 08:10:47 SRVName kernel: [410744.629015] st0: Sense Key : Unit Attention
[current]
Jun 4 08:10:47 SRVName kernel: [410744.629026] st0: Add. Sense: Power on,
reset, or bus device reset occurred
Jun 4 08:14:02 SRVName kernel: [410939.819168] st0: Sense Key : Unit Attention
[current]
Jun 4 08:14:02 SRVName kernel: [410939.819180] st0: Add. Sense: Power on,
reset, or bus device reset occurred
Jun 4 08:16:57 SRVName kernel: [44.538975] st0: Sense Key : Unit Attention
[current]
Jun 4 08:16:57 SRVName kernel: [44.538988] st0: Add. Sense: Power on,
reset, or bus device reset occurred
===END SYSLOG
==
Googling for those entries I found
http://bacula.10910.n7.nabble.com/Bacula-tapes-marked-FULL-too-early-VolBytes-too-low-td58881i20.html.
Similar issue (but no report of tar), the thread ended with "similar problem
went away with replaced drive" & "get your drive tested"
>From the Bacula log ("Error: Re-read of last block OK, but block numbers
>differ. Read block=990557 Want block=990558.") it looks like Bacula checks up
>on what has been written every so often. I don't think tar does that; it just
>streams to tape. If my card/cable/tape is only slightly flaky, is it
>reasonable to think that this extra work pushes it over the edge? Or am I
>barking up the wrong tree?
Thanks,
Dan Stieneke
- from Dan Langille -
If it is all tapes, is the issue with the tape drive?
- from Martin Simmons -
Check the syslog and system console for error messages about the tape device
(since Bacula saw Input/output error, that usually means some error on the
device).
>>>>> On Thu, 7 Jun 2018 15:38:13 +, Stieneke, Dan said:
>
> The job ate through 4 tapes, with only 2 - 60GB on each tape. Then it hit
> recycle limits and was asking for more media.
>
> These are used tapes, but I can't see 4 consecutive tapes going bad at the
> same time.
>
> Incidentally, this is the same behavior I saw 4 months ago, and at that time
> I did test bacula to a brand-new tape, which also failed quickly.
>
> Thanks,
> Dan
>
>
> From: Josh Fisher [mailto:jfis...@pvct.com]
> Sent: Wednesday, June 6, 2018 5:18 AM
> To: Stieneke, Dan ;
> 'bacula-users@lists.sourceforge.net'
>
> Subject: Re: [Bacula-users] Bacula h/w write fails, but tar writes w/out
> error?
>
>
> On 6/5/2018 3:45 PM, Stieneke, Dan wrote:
> Ubuntu 16.04, Bacula 5.2.6, single-drive autoloader, all running Bacula
> trouble-free for years.
>
> Four months ago I got some errors in Bacula that looked like h/w errors,
> although jobs using tar on the same drive ran without error. I had suspicions
> about a cable, and when I replaced it everything returned to normal, until
> now, when I'm getting the same kinds of errors.
>
> Tar works on the same drive, but what about on the same tape? How do you know
> you are not seeing bad tapes?
>
>
>
> The relevant part of "messages" is:
> = = = = = = = = = = = = = = = = = =
> 05-Jun 09:17 xxx-sd JobId 794: Error: block.c:577 Write error at 12:60511 on
> device "Ultrium-TD4" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD4_1310010391-nst).
> ERR=Input/output error.
> 05-Jun 09:18 xxx-sd JobId 794: Error: Re-read of last block OK, but block
> numbers differ. Read block=990557 Want block=990558.
> 05-Jun 09:18 xxx-sd JobId 794: End of medium on Volume "A00030L4"
> Bytes=63,902,942,208 Blocks=990,558 at 05-Jun-2018 09:18.
> 05-Jun 09:18 xxx-sd JobId 794: 3307 Issuing autochanger "unload slot 16,
> drive 0" command.
> = = = = = = = = = = = = = = = = = =
>
> As you can see, it had an error after about 64GB (of an 800GB native / 1600GB
> compressed tape).
>
> I've cleaned the drive. And again, backups made with tar record without error
> and restore without error.
> Any ideas?
>
> Thanks,
> Dan Stieneke
> IT Specialist
> USDA - ARS - NWISRL
> 3793 N 3600 E
> Kimberly, ID 83341
>
>
>
>
> This electronic message contains information generated by the USDA solely for
> the intended recipients. Any unauthorized interception of this message or the
> use or disclosure of the information it contains may viol
Re: [Bacula-users] Bacula h/w write fails, but tar writes w/out error?
The job ate through 4 tapes, with only 2 - 60GB on each tape. Then it hit recycle limits and was asking for more media. These are used tapes, but I can't see 4 consecutive tapes going bad at the same time. Incidentally, this is the same behavior I saw 4 months ago, and at that time I did test bacula to a brand-new tape, which also failed quickly. Thanks, Dan From: Josh Fisher [mailto:jfis...@pvct.com] Sent: Wednesday, June 6, 2018 5:18 AM To: Stieneke, Dan ; 'bacula-users@lists.sourceforge.net' Subject: Re: [Bacula-users] Bacula h/w write fails, but tar writes w/out error? On 6/5/2018 3:45 PM, Stieneke, Dan wrote: Ubuntu 16.04, Bacula 5.2.6, single-drive autoloader, all running Bacula trouble-free for years. Four months ago I got some errors in Bacula that looked like h/w errors, although jobs using tar on the same drive ran without error. I had suspicions about a cable, and when I replaced it everything returned to normal, until now, when I'm getting the same kinds of errors. Tar works on the same drive, but what about on the same tape? How do you know you are not seeing bad tapes? The relevant part of "messages" is: = = = = = = = = = = = = = = = = = = 05-Jun 09:17 xxx-sd JobId 794: Error: block.c:577 Write error at 12:60511 on device "Ultrium-TD4" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD4_1310010391-nst). ERR=Input/output error. 05-Jun 09:18 xxx-sd JobId 794: Error: Re-read of last block OK, but block numbers differ. Read block=990557 Want block=990558. 05-Jun 09:18 xxx-sd JobId 794: End of medium on Volume "A00030L4" Bytes=63,902,942,208 Blocks=990,558 at 05-Jun-2018 09:18. 05-Jun 09:18 xxx-sd JobId 794: 3307 Issuing autochanger "unload slot 16, drive 0" command. = = = = = = = = = = = = = = = = = = As you can see, it had an error after about 64GB (of an 800GB native / 1600GB compressed tape). I've cleaned the drive. And again, backups made with tar record without error and restore without error. Any ideas? Thanks, Dan Stieneke IT Specialist USDA - ARS - NWISRL 3793 N 3600 E Kimberly, ID 83341 This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net<mailto:Bacula-users@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/bacula-users -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Bacula h/w write fails, but tar writes w/out error?
Ubuntu 16.04, Bacula 5.2.6, single-drive autoloader, all running Bacula trouble-free for years. Four months ago I got some errors in Bacula that looked like h/w errors, although jobs using tar on the same drive ran without error. I had suspicions about a cable, and when I replaced it everything returned to normal, until now, when I'm getting the same kinds of errors. The relevant part of "messages" is: = = = = = = = = = = = = = = = = = = 05-Jun 09:17 xxx-sd JobId 794: Error: block.c:577 Write error at 12:60511 on device "Ultrium-TD4" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD4_1310010391-nst). ERR=Input/output error. 05-Jun 09:18 xxx-sd JobId 794: Error: Re-read of last block OK, but block numbers differ. Read block=990557 Want block=990558. 05-Jun 09:18 xxx-sd JobId 794: End of medium on Volume "A00030L4" Bytes=63,902,942,208 Blocks=990,558 at 05-Jun-2018 09:18. 05-Jun 09:18 xxx-sd JobId 794: 3307 Issuing autochanger "unload slot 16, drive 0" command. = = = = = = = = = = = = = = = = = = As you can see, it had an error after about 64GB (of an 800GB native / 1600GB compressed tape). I've cleaned the drive. And again, backups made with tar record without error and restore without error. Any ideas? Thanks, Dan Stieneke IT Specialist USDA - ARS - NWISRL 3793 N 3600 E Kimberly, ID 83341 This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] mbuffer as tool for concurrent spool/despool
Reading the mail archive, it seems I'm not alone in wanting to see concurrent
spooling/despooling.
Could we hand off the job to a standalone spooler like mbuffer
(http://www.maier-komor.de/mbuffer.html)? For a simple tar to LTO-4 job it
makes a tremendous improvement.
The Bacuala developers would need to modify the storage daemon to allow the
datastream to be diverted out and then back in through a pair of FIFOs before
it gets [properly re-blocked? and] written to tape. The user would need to tie
the FIFOs together with an external spooler. It might look like this:
--/etc/bacula-sd.conf--
snip
Device {
...
PushToFIFOSpooler = /path/to/bufreceive# bufreceive is a FIFO on this
machine
PullFromFIFOSpooler = /path/to/bufsend # bufsend is also a FIFO
...
}
/snip
--/etc/bacula-dir.conf--
snip
Job {
...
RunBeforeJob = mbuffer -i /path/to/bufreceive -o /path/to/bufsend # default
puny 2M RAM buffer; can be set to multi-gig RAM or disk
# Alternatively: RunBeforeJob = mbuffer /path/to/bufreceive
/path/to/bufsend
...
}
/snip
This would
- be a lot easier than asking the developers to reinvent the wheel
(insert your own ring-buffer pun here...)
- let the developers all but ignore spooling focus on real backup
stuff
- give users access to many more options, such as using RAM or disk
spool, high/low-watermarks, and rate-limiting
- let folks cleverer than me figure out other cool things to do with
the datastream
Thanks,
Dan Stieneke
IT Specialist
USDA - ARS - NWISRL
3793 N 3600 E
Kimberly, ID 83341
208/423-6519
This electronic message contains information generated by the USDA solely for
the intended recipients. Any unauthorized interception of this message or the
use or disclosure of the information it contains may violate the law and
subject the violator to civil or criminal penalties. If you believe you have
received this message in error, please notify the sender and delete the email
immediately.
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
