Amanda didn't take new tape
Hello,
I use an Overland PowerLoader (16 tapes) with an LTO-2 device. I do a
full backup two times a week.
I did an upgrade from amanda 2.4.4p3-3 (Debian sarge) to Amanda
2.5.2p1-4 (Debian lenny).
After this upgrade amanda didn't take a new tape if a DLE failed because
of emty space on the current tape.
I changed some things of the amanda.conf because they changed in the new
version. amanda is working but didn't change the tape and start the DLE
with the new.
The error message in the amanda status report is:
dump to tape failed: driver: [writing file: No space left on device]
I got the same error with the old amanda but then amanda take a new tape
and start to backup the DLE that wasn't finished correct.
Any advice for me that I missed to change?
Here is my current amanda.conf (2.5.2p1-4):
org XX -
mailto x...@.xxx
dumpuser backup
inparallel 10
dumporder SssSssSssSssSssSssSssS
taperalgo largest
netusage 1 Kbps
dumpcycle 0
runspercycle 28
tapecycle 290 tapes
bumpsize 20 Mb
bumppercent 20
bumpdays 1
bumpmult 4
etimeout 300
dtimeout 1800
ctimeout 30
tapebufs 20
runtapes 7
tpchanger chg-zd-mtx
tapedev /dev/nst0
changerfile /etc/amanda/X/CHANGER
changerdev /dev/sg2
maxdumpsize -1
tapetype X_LTO2
labelstr ^X-[A-Z][A-Z][A-Z][0-9][0-9][0-9][A-Z][0-9]*$
amrecover_do_fsf yes
amrecover_check_label yes
amrecover_changer /dev/sg2
autoflush no
infofile /var/lib/amanda/X/curinfo
logdir /var/lib/amanda/X/log
indexdir /var/lib/amanda/X/index
tapelist /etc/amanda/DailyFull/tapelist
define tapetype X_LTO2 {
comment LTO2 HP Ultrium2 (hardware compression on)
length 195000 mbytes
filemark 0 kbytes
speed 25000 kbytes
}
define dumptype global {
comment Global definitions
auth bsd
}
define dumptype X-full {
global
comment Always full dump of all date
program GNUTAR
compress none
priority high
dumpcycle 0
index
record no
skip-incr yes
}
--
Best regards
Dominik Schips
Re: [Amanda-users] Advice needed on Linux backup strategy to LTO-4 tape
Rory Campbell-Lange wrote: On 14/08/09, Frank Smith (fsm...@hoovers.com) wrote: Chris Hoogendyk wrote: Amanda will do the compression for you. You define it in the dumptype in amanda.conf. If you have a holding disk, then it will compress the data as it goes onto the holding disk. If you don't have a holding disk, then you might have issues with being able to stream a backup to tape, compressing it on the fly. Even with a really fast cpu, I don't know if you can maintain the throughput to drive LTO4 at a good speed. You might want to consider configuring for client compression. Not only will that give you more CPU for feeding your tape, it also minimizes network bandwidth. As usual, YMMV, it all depends on where the bottlenecks are in your environment. In our case the server _is_ the only client, with up to 30TB of direct attached storage, with the storage running at between 80MB/s and 120MB/s access speeds (Bytes rather than bytes). I don't know if this is fast enough to deal with a SAS connected LTO4 drive, particularly if it is doing software compression along the way. With reference to Chris Hoogendyk's email clarification on parallelism, I am very curious to learn if Amanda ...still require[s] a DLE to be completed to holding disk before it will send any of it to tape... In our case this is a particularly important question as, although we can add in more AoE storage for a DLE, this will only run at the speeds above. Do we need a 1TB SAS disk array too? You will get the best performance if you can do that. If the disk that is being copied to tape can give the speed the tape needs, that's going to do a better job of keeping things moving. You have a couple of options. You can go without a holding disk, and then each DLE will be streamed sequentially to tape. This will stretch out your backups. It will also mean that any compression you do in software will be done in line with that sequential stream. Your system may not be able to keep that all flying fast enough for the tape, and you may end up with shoe shining and very low speeds. You can certainly try it and see what happens. If (when?) that fails, you could try using hardware compression on the tape drive. The backups will still be sequential, one DLE streaming to tape at a time, and if your drives can't keep up, it will be slower than you might like. But, at least you are not dealing with network backups. The option I would try, budget allowing, would be to add a couple of SAS drives to be used as holding disks. Then break up your DLEs so that each DLE is substantially smaller than the holding disks. Then Amanda can run them in parallel, compress them on the holding disk, and then stream completed, compressed DLEs from the holding disk to the tape. I wouldn't put the holding disks in raid. -- --- Chris Hoogendyk - O__ Systems Administrator c/ /'_ --- Biology Geology Departments (*) \(*) -- 140 Morrill Science Center ~~ - University of Massachusetts, Amherst hoogen...@bio.umass.edu --- Erdös 4
Re: [Amanda-users] Advice needed on Linux backup strategy to LTO-4 tape
I wouldn't put the holding disks in raid. Hu hu... Interesting... I have a 4 disks RAID-0 holding disk, and it isn't fast... I always wondered if I should use seperated (non-RAID) drives... Cyrille Bollu Responsable systèmes Fedasil - ICT tel: +32.2.213.43.49 gsm: +32.478.23.08.15
Re: [Amanda-users] Advice needed on Linux backup strategy to LTO-4 tape
On Monday 17 August 2009, Cyrille Bollu wrote: I wouldn't put the holding disks in raid. Hu hu... Interesting... I have a 4 disks RAID-0 holding disk, and it isn't fast... I always wondered if I should use seperated (non-RAID) drives... Its been my observation that software raids are slower. Not tremendously so though. When Jim put together a raid-5 with 4 drives several years ago, the drives were about 70meg/sec drives, and the overall was just a hair over 50meg/sec. I know he has rebuilt it with bigger faster drives 2 or 3 times since, along with more iron in the cpu, so I don't know its current speed. Being retired means being out of the loop. :( Cyrille Bollu Responsable systèmes Fedasil - ICT tel: +32.2.213.43.49 gsm: +32.478.23.08.15 -- Cheers, Gene There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. -Ed Howdershelt (Author) The NRA is offering FREE Associate memberships to anyone who wants them. https://www.nrahq.org/nrabonus/accept-membership.asp A prig is a fellow who is always making you a present of his opinions. -- George Eliot
Re: Backup issues with OpenBSD 4.5 machines
On Fri, Aug 14, 2009 at 03:26:10PM -0400, Jean-Louis Martineau wrote: Check system log Post complete amandad.*.debug and sendbackup.*.debug. You can try to disable client compression. It still fails with client side compression truned off. -- One of the main causes of the fall of the roman empire was that, lacking zero, they had no way to indicate successful termination of their C programs.
Re: Backup issues with OpenBSD 4.5 machines
And what is the error? It's probably not a gzip: ... Jean-Louis stan wrote: On Fri, Aug 14, 2009 at 03:26:10PM -0400, Jean-Louis Martineau wrote: Check system log Post complete amandad.*.debug and sendbackup.*.debug. You can try to disable client compression. It still fails with client side compression truned off.
Re: [Amanda-users] Advice needed on Linux backup strategy to LTO-4 tape
On Monday 17 August 2009, Cyrille Bollu wrote: I wouldn't put the holding disks in raid. Hu hu... Interesting... I have a 4 disks RAID-0 holding disk, and it isn't fast... I always wondered if I should use seperated (non-RAID) drives... To drive an LTO-4 your holding disk needs to read somewhat over 100MB/sec sequentially, which requires at least 2 drives striped, but should be easy enough with any modern raid controller or software raid. Complicating this, it may also need to write at similar speed, simultaneously, which introduces a random access element and ups the demand considerably. I would think you would probably want at least 4 big SATA drives striped together to reliably feed an LTO-4 drive at full speed. Conveniently, this could also give you over 5TB of very cheap holding disk space. Also, unless you're backing up exclusively large files over a fast SAN link (faster than Gig-E), I doubt you could get anywhere close to full tape performance without a holding disk. -- ... a serious depression seems improbable; [we expect] recovery of business next spring, with further improvement in the fall. Harvard Economic Society, November 10, 1929
Re: Backup issues with OpenBSD 4.5 machines
On Mon, Aug 17, 2009 at 11:30:51AM -0400, Jean-Louis Martineau wrote: And what is the error? It's probably not a gzip: ... Interesting. I set up a test doing just one of these machine (4 DLE's), and it worked in non-compressed. I will leave a couple of the OpenBSD machines set fro non-compressed in tonights ru, and see what ahppens. Thanks. -- One of the main causes of the fall of the roman empire was that, lacking zero, they had no way to indicate successful termination of their C programs.
Re: [Amanda-users] Advice needed on Linux backup strategy to LTO-4 tape
Cyrille Bollu wrote: Chris Hoogendyk wrote: I wouldn't put the holding disks in raid. Hu hu... Interesting... I have a 4 disks RAID-0 holding disk, and it isn't fast... I always wondered if I should use seperated (non-RAID) drives... Here is an extremely interesting article that everyone should take a look at. Rather than just giving theoretical comparisons that you see in places like the Wikipedia raid article (which is nevertheless an excellent reference), this article is a case study analysis with both real and test environments throwing data at raid and instrumenting it -- http://blogs.zdnet.com/Ou/?p=484. While this guy is looking at things like database servers and exchange, we ought to be able to interpret this for Amanda. A couple of points to note for Amanda: Amanda will use all the holding disk drives while doing parallel backups and storing output on the holding disks. When it is writing to tape, it is constrained by the sequential nature of the tape, and will only be doing one DLE at a time from those that it has completed on the holding disks. Also, Amanda's access is heavily sequential, although it may have multiple parallel processes hitting the drives. -- --- Chris Hoogendyk - O__ Systems Administrator c/ /'_ --- Biology Geology Departments (*) \(*) -- 140 Morrill Science Center ~~ - University of Massachusetts, Amherst hoogen...@bio.umass.edu --- Erdös 4
Re: [Amanda-users] Advice needed on Linux backup strategy to LTO-4 tape
On 17/08/09, Chris Hoogendyk (hoogen...@bio.umass.edu) wrote: Cyrille Bollu wrote: Chris Hoogendyk wrote: I wouldn't put the holding disks in raid. snip http://blogs.zdnet.com/Ou/?p=484. While this guy is looking at things like database servers and exchange, we ought to be able to interpret this for Amanda. A couple of points to note for Amanda: Amanda will use all the holding disk drives while doing parallel backups and storing output on the holding disks. When it is writing to tape, it is constrained by the sequential nature of the tape, and will only be doing one DLE at a time from those that it has completed on the holding disks. Also, Amanda's access is heavily sequential, although it may have multiple parallel processes hitting the drives. A slow RAID1 off two 7200 RPM SATA disks on a BBU-backed LSI hardware raid controller can do about 62031 KiB/s write and 86399 KiB/s read. Those sorts of numbers improve steadily the number of spindles you add to a RAID collection and the higher the RAID number and (in the case of writing) if cacheing is enabled. On 16/08/09, Rory Campbell-Lange (r...@campbell-lange.net) wrote: On 14/08/09, Frank Smith (fsm...@hoovers.com) wrote: Chris Hoogendyk wrote: With reference to Chris Hoogendyk's email clarification on parallelism, I am very curious to learn if Amanda ...still require[s] a DLE to be completed to holding disk before it will send any of it to tape... In our case this is a particularly important question as, although we can add in more AoE storage for a DLE, this will only run at the speeds above. Do we need a 1TB SAS disk array too? From the discussion here it seems preferable to have a DLE on two major counts. One is that compression can happen prior to writing to tape, which could result in shoe-shining, and another is that Amanda will be clearer about the amount of data it will be trying to write to a tape, in other words it will do a better fit of data to tape. The most important question I now have to ask is: How fast can a SAS-based LTO4 drive write to tape? Regards Rory -- Rory Campbell-Lange Director r...@campbell-lange.net Campbell-Lange Workshop www.campbell-lange.net 0207 6311 555 3 Tottenham Street London W1T 2AF Registered in England No. 04551928
