Re: filesystem limit?
On Wed, 17 Mar 2004 at 8:02am, Geoff Swavley wrote I was wondering if anyone could tell me why amanda seems to split my filesystem into 2 holding files? An error has occurred so these filesare This is normal, and the size of the chunks are set by the 'chunksize' parameter in amanda.conf. NOTES: taper: tape schedule8-WEEK3 kb 31851520 fm 1 writing file: short write driver: going into degraded mode because of tape error. How big are your tapes? It looks like you hit EOT here... -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: filesystem limit?
35GB AIT-1 tapes (Native) plus I also have compression on the
drives DISABLED (advice from the list about double compressing a file,
resulting in a larger file than what you started with). It has slowed down
the backups but the large (already compressed thanks to oracle) export
database files seem to fit on the tape now.
Those two files add up as follows:
2355658752+
32212254720
---
34567913472
---
and the filesystem is 33.7GB so it should just fit on:
-
HOST FILESYSTEM TOTAL USED
pokolbin /u2036144364 33701475
-
TOTALS: 36144364 33701475
Maybe I'm living too close to the edge? Maybe the tape is bad towards the end ...
I'll change the tape and try flushing again. Hm ... that's
3 tapes now and all with the same flush error:
Subject: schedule8 AMFLUSH MAIL REPORT FOR March 17, 2004
Date: Wed, 17 Mar 2004 09:28:57 +1100 (EST)
From: Amanda Archiving Server [EMAIL PROTECTED]
To: [EMAIL PROTECTED], [EMAIL PROTECTED]
The dumps were flushed to tape schedule8-WEEK3.
*** A TAPE ERROR OCCURRED: [[writing file: I/O error]].
Some dumps may have been left in the holding disk.
Run amflush again to flush them to tape.
The next tape Amanda expects to use is: schedule8-WEEK4.
FAILURE AND STRANGE DUMP SUMMARY:
pokolbin /u20 lev 0 FAILED [out of tape]
STATISTICS:
Total Full Daily
Estimate Time (hrs:min)0:00
Run Time (hrs:min) 1:10
Dump Time (hrs:min)0:00 0:00 0:00
Output Size (meg) 0.00.00.0
Original Size (meg) 0.00.00.0
Avg Compressed Size (%) -- -- --
Filesystems Dumped0 0 0
Avg Dump Rate (k/s) -- -- --
Tape Time (hrs:min)0:00 0:00 0:00
Tape Size (meg) 0.00.00.0
Tape Used (%) 0.00.00.0
Filesystems Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --
USAGE BY TAPE:
Label Time Size %Nb
schedule8-WEEK3 0:00 0.00.0 0
NOTES:
taper: tape schedule8-WEEK3 kb 3440992 fm 1 writing file: I/O error
DUMP SUMMARY:
DUMPER STATSTAPER STATS
HOSTNAME DISKL ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s
-- -
pokolbin /u200 N/A 0 --N/A N/A FAILED
(brought to you by Amanda version 2.4.4p2)
Joshua Baker-LePain wrote:
On Wed, 17 Mar 2004 at 8:02am, Geoff Swavley wrote
I was wondering if anyone could tell me why amanda seems to split
my filesystem into 2 holding files? An error has occurred so these filesare
This is normal, and the size of the chunks are set by the 'chunksize'
parameter in amanda.conf.
NOTES:
taper: tape schedule8-WEEK3 kb 31851520 fm 1 writing file: short write
driver: going into degraded mode because of tape error.
How big are your tapes? It looks like you hit EOT here...
--
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University
geoffs :-)
--
Geoff Swavley Email : [EMAIL PROTECTED]
UNIX Sys Admin Snail : Level 8, 10 Valentine Ave,
Support and Network Operations Parramatta NSW 2150
Dept of Infrastructure, PlanningSydney Australia
and Natural Resources Phone : 61-2-98957125
http://www.radx.net/~geoffs Fax : 61-2-98957086
Mobile: 61-422-002005 Home : 61-2-96593637
--
Be wary of the man who urges an action in which he himself incurs
no risk. - Setanti, Joaquin de
Re: filesystem limit?
On Tuesday 16 March 2004 16:02, Geoff Swavley wrote:
Hi All,
I was wondering if anyone could tell me why amanda seems to split
my filesystem into 2 holding files? An error has occurred so these
filesare left on the disk. Here is the filesystem listing:
at the files:
465921 drwxr-xr-x 4 amanda sys 512 Mar 17 00:45
/holding2/schedule8
2096641 drwx-- 2 amanda sys 512 Mar 13 05:22
/holding2/schedule8/20040312
209666 2301584 -rw--- 1 amanda sys 2355658752 Mar 13
05:22 /holding2/schedule8/20040312/pokolbin._u20.0.1
209665 31472648 -rw--- 1 amanda sys 32212254720 Mar 13
05:07 /holding2/schedule8/20040312/pokolbin._u20.0
2038401 drwx-- 2 amanda sys 512 Mar 17 00:45
/holding2/schedule8/20040317
Has it got something to do with a 32GB limit? What does the .1
and .0 refer to?
Obviously the .1 ias the 2nd file of the filesystem??
thanks all. Here is the error email I received:
Subject: schedule8 AMANDA MAIL REPORT FOR March 12, 2004
Date:Sat, 13 Mar 2004 08:26:54 +1100 (EST)
From: Amanda Archiving Server [EMAIL PROTECTED]
To:[EMAIL PROTECTED], [EMAIL PROTECTED]
These dumps were to tape schedule8-WEEK3.
*** A TAPE ERROR OCCURRED: [[writing file: short write]].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: schedule8-WEEK4.
FAILURE AND STRANGE DUMP SUMMARY:
pokolbin /u20 lev 0 FAILED [out of tape]
STATISTICS:
Total Full Daily
Estimate Time (hrs:min)0:00
Run Time (hrs:min) 9:41
Dump Time (hrs:min)6:34 6:34 0:00
Output Size (meg) 32966.532966.50.0
Original Size (meg) 32966.532966.50.0
Avg Compressed Size (%) -- -- --
Filesystems Dumped1 1 0
Avg Dump Rate (k/s) 1426.7 1426.7--
Tape Time (hrs:min)0:00 0:00 0:00
Tape Size (meg) 0.00.00.0
Tape Used (%) 0.00.00.0
Filesystems Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --
USAGE BY TAPE:
Label Time Size %Nb
schedule8-WEEK3 0:00 0.00.0 0
NOTES:
taper: tape schedule8-WEEK3 kb 31851520 fm 1 writing file: short
write driver: going into degraded mode because of tape error.
DUMP SUMMARY:
DUMPER STATSTAPER
STATS HOSTNAME DISKL ORIG-KB OUT-KB COMP% MMM:SS KB/s
MMM:SS KB/s --
- pokolbin /u20
0 3375766433757664 -- 394:211426.7 FAILED
(brought to you by Amanda version 2.4.4p2)
This looks to me as if you ran out of tape. Did you just add these to
the disklist? How did you determine the tapetype? Is the tape
drives hardware compressor enabled? Amtapetype can determine this
and tell you.
The latter case in particular can and will hide the true capacity of a
tape from amanda. Please understand that amanda counts bytes sent to
the drive when making its judgements as to tape fitting. Enabling
the drives compressor makes amanda think the tape is much larger than
it really is so this users recommendation is and always has been to
shut off any hardware compression and use only software compression.
Yes, it takes time and cpu horsepower to do software compression, but
it can beat the socks off the hardware compressor for most
compressable data, sometimes smunching things down to 10% of the
original byte count. Since amanda tracks the size of the compressed
file, amanda then knows exactly how much has been moved to the media
and very very rarely will amanda hit an EOT then.
When adding a new path to amanda's disklist, I always set the
compression on, then look at the mailed report from the next run. If
compression only saved 5 or 10 percent, then that data in that path
should be considered as already compressed, like a directory full of
rpm's or tar.gz's. Binary executables generally will fall into this
category too, so one shouldn't waste the cpu horsepower to even try.
OTOH, your /etc dir will probably smunch to 20% of its original size
and should be compressed on most systems.
Also, keep in mind that amanda, generally speaking, cannot handle a
disklist entry thats truely bigger than the tape because amanda
cannot, without patching, span a single entry across more than one
tape. Because that involves tape operations that amanda may not have
full control over, thereby endangering the data, that limitation
isn't something the authors are working to fix as its not really
considered broken by anyone except the fellow who bought a too small
tape drive.
Disklist entries can be
Re: filesystem limit?
On Tuesday 16 March 2004 17:51, Geoff Swavley wrote:
35GB AIT-1 tapes (Native) plus I also have compression on the
drives DISABLED (advice from the list about double compressing a
file, resulting in a larger file than what you started with). It
has slowed down the backups but the large (already compressed
thanks to oracle) export database files seem to fit on the tape
now.
Those two files add up as follows:
2355658752+
32212254720
---
34567913472
---
Thats rather close. How many bytes is a filemark on that tape?
and the filesystem is 33.7GB so it should just fit on:
-
HOST FILESYSTEM TOTAL USED
pokolbin /u2036144364 33701475
-
TOTALS: 36144364 33701475
Maybe I'm living too close to the edge? Maybe the tape is bad
towards the end ... I'll change the tape and try flushing again.
Hm ... that's 3 tapes now and all with the same flush error:
Subject: schedule8 AMFLUSH MAIL REPORT FOR March 17, 2004
Date: Wed, 17 Mar 2004 09:28:57 +1100 (EST)
From: Amanda Archiving Server [EMAIL PROTECTED]
To: [EMAIL PROTECTED], [EMAIL PROTECTED]
The dumps were flushed to tape schedule8-WEEK3.
*** A TAPE ERROR OCCURRED: [[writing file: I/O error]].
Some dumps may have been left in the holding disk.
Run amflush again to flush them to tape.
The next tape Amanda expects to use is: schedule8-WEEK4.
FAILURE AND STRANGE DUMP SUMMARY:
pokolbin /u20 lev 0 FAILED [out of tape]
STATISTICS:
Total Full Daily
Estimate Time (hrs:min)0:00
Run Time (hrs:min) 1:10
Dump Time (hrs:min)0:00 0:00 0:00
Output Size (meg) 0.00.00.0
Original Size (meg) 0.00.00.0
Avg Compressed Size (%) -- -- --
Filesystems Dumped0 0 0
Avg Dump Rate (k/s) -- -- --
Tape Time (hrs:min)0:00 0:00 0:00
Tape Size (meg) 0.00.00.0
Tape Used (%) 0.00.00.0
Filesystems Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --
USAGE BY TAPE:
Label Time Size %Nb
schedule8-WEEK3 0:00 0.00.0 0
NOTES:
taper: tape schedule8-WEEK3 kb 3440992 fm 1 writing file: I/O
error
DUMP SUMMARY:
DUMPER STATSTAPER
STATS HOSTNAME DISKL ORIG-KB OUT-KB COMP% MMM:SS KB/s
MMM:SS KB/s --
- pokolbin /u20
0 N/A 0 --N/A N/A FAILED
(brought to you by Amanda version 2.4.4p2)
Joshua Baker-LePain wrote:
On Wed, 17 Mar 2004 at 8:02am, Geoff Swavley wrote
I was wondering if anyone could tell me why amanda seems to
split my filesystem into 2 holding files? An error has
occurred so these filesare
This is normal, and the size of the chunks are set by the
'chunksize' parameter in amanda.conf.
NOTES:
taper: tape schedule8-WEEK3 kb 31851520 fm 1 writing file:
short write driver: going into degraded mode because of tape
error.
How big are your tapes? It looks like you hit EOT here...
--
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University
geoffs :-)
-- Geoff Swavley Email : [EMAIL PROTECTED] UNIX
Sys Admin Snail : Level 8, 10 Valentine Ave,
Support and Network Operations Parramatta NSW 2150
Dept of Infrastructure, PlanningSydney Australia and
Natural Resources Phone : 61-2-98957125
http://www.radx.net/~geoffs Fax : 61-2-98957086
Mobile: 61-422-002005 Home : 61-2-96593637
---
--- Be wary of the man who urges an action in which he himself
incurs no risk. - Setanti, Joaquin de
--
Cheers, Gene
There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order.
-Ed Howdershelt (Author)
99.22% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attornies please note, additions to this message
by Gene Heskett are:
Copyright 2004 by Maurice Eugene Heskett, all rights reserved.
Re: filesystem limit?
Geoff Swavley wrote: I was wondering if anyone could tell me why amanda seems to split my filesystem into 2 holding files? An error has occurred so these filesare It sounds like you have other, unrelated problems, but check the setting of chunksize in amanda.conf, that is usually what determines dumps getting split up into multiple holding files. In any case, it is not determined by the filesystem--you can give amanda a chunksize that is too large for some filesystem type, in which case it will keep writing until it gets an I/O error. The holding files are concatenated during writing to tape, at which point the holding file size becomes irrelevant. Most modern filesystems can support file sizes up to the TB range--I'm using a chunksize of 256GB on Linux XFS, it could be larger but Why go larger than the size of the holding disk? Exceptions include (but are not limited to) ext2, SGI efs, iso9660, FAT(12|16|32), NFSv2, not sure about ufs, hfs, NTFS, I'm sure there are others. 2 GB and 8 GB are typical limits. But as I said AFAIK amanda will not automagically set the chunksize for the holding disk filesystem type, it would just keep writing until it got an I/O error, but then there have been a lot of changes from 2.4.2 to 2.4.4 that I am just catching up with now. --jonathan
