Hi,
I've recently seen Amanda fail to write a dump to tape fully (write
error), but I wasn't prompted to run amflush to flush the remaining
files to tape, I usually am when there is data in the holding disk.
I'm not sure how this could happen, and this dump had several special
conditions. Anyways, I have configured amanda to flush the leftover
files at the next run. So she did, and hence there was way too much data
for a tape and it failed again.
>2.5 GB in the holding disk and the tape manages about 3.2 GB).
I am attaching the notification mails in order, host names have been
normalized and uninteresting gtar complaints ("socket ignored") omitted.
The interesting questions:
1. why did not amanda prompt me to run amflush on the first run, if
there was a tape write error?
2. why did not amanda massively bump to higher dump levels when there
was a file in the holding disk that had to be flushed first?
Because someone forgot to write the necessary code? :-)
Ultimately, a manual amflush run cleared everything up but I am under
the impression amanda should have asked me at the first dump to run
amflush. There has been a window of two days and the backed up file
system and the holding disk are from/on the same host. (I might not have
cared even half as much if the holding disk were in a different host.)
TIA,
--
Matthias Andree
Encrypt your mail: my GnuPG key ID is 0x052E7D95
These dumps were to tape FullBacks6.
The next tape Amanda expects to use is: FullBacks2.
FAILURE AND STRANGE DUMP SUMMARY:
planner: ERROR Request to host02 timed out.
host01 /var lev 1 STRANGE
host01 /home lev 0 FAILED [input: Can't read data: : Input/output error]
host02 /home RESULTS MISSING
host02 /etc RESULTS MISSING
host02 /var RESULTS MISSING
STATISTICS:
Total Full Daily
-------- -------- --------
Estimate Time (hrs:min) 0:10
Run Time (hrs:min) 1:44
Dump Time (hrs:min) 0:23 0:21 0:03
Output Size (meg) 2952.4 2684.5 267.9
Original Size (meg) 2952.4 2684.5 267.9
Avg Compressed Size (%) -- -- -- (level:#disks ...)
Filesystems Dumped 8 1 7 (1:7)
Avg Dump Rate (k/s) 2152.7 2194.5 1807.7
Tape Time (hrs:min) 0:09 0:00 0:09
Tape Size (meg) 268.0 0.0 268.0
Tape Used (%) 8.8 0.0 8.8 (level:#disks ...)
Filesystems Taped 7 0 7 (1:7)
Avg Tp Write Rate (k/s) 481.8 -- 481.8
USAGE BY TAPE:
Label Time Size % Nb
FullBacks6 0:09 268.0 8.8 7
FAILED AND STRANGE DUMP DETAILS:
/-- host01 /var lev 1 STRANGE
sendbackup: start [host01:/var level 1]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/tar -f... -
sendbackup: info end
? gtar: ./lib/amanda/FullBacks/index/host03/_var/20040214_1.gz.tmp: Warning: Cannot
stat: No such file or directory
| Total bytes written: 154603520 (147MB, 2.2MB/s)
sendbackup: size 150980
sendbackup: end
\--------
NOTES:
planner: Incremental of host01:/home bumped to level 2.
driver: going into degraded mode because of tape error.
taper: tape FullBacks6 kb 1738048 fm 8 [OK]
DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s
-------------------------- --------------------------------- ------------
host03 / 1 100 128 -- 0:08 13.1 0:01 156.0
host03 /usr 1 12190 12192 -- 0:15 799.6 0:17 706.0
host03 /var 1 6470 6496 -- 0:08 789.2 0:10 643.5
host01 / 1 94300 94304 -- 0:462029.2 5:08 305.8
host01 /home 0 27489702748970 -- 20:532194.5 FAILED ----
host01 /space/ICD 1 340 352 -- 0:01 669.5 0:01 364.1
host01 /usr/local 1 9900 9920 -- 0:051849.8 0:19 514.0
host01 /var 1 150980 151008 -- 1:082209.9 3:33 709.9
host02 /etc MISSING --------------------------------------
host02 /home MISSING --------------------------------------
host02 /var MISSING --------------------------------------
(brought to you by Amanda version 2.4.4)
These dumps were to tape FullBacks2.
*** A TAPE ERROR OCCURRED: [[writing file: No space left on device]].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: FullBacks4.
FAILURE AND STRANGE DUMP SUMMARY:
host01 /var lev 1 STRANGE
host02 /home lev 0 FAILED [out of tape]
STATISTICS:
Total Full Daily
-------- -------- --------
Estimate Time (hrs:min) 0:10
Run Time (hrs:min) 2:41
Dump Time (hrs:min) 0:38 0:27 0:11
Output Size (meg) 2799.0 2346.0 453.0
Original Size (meg) 2799.0 2346.0 453.0
Avg Compressed Size (%) -- -- -- (level:#disks ...)
Filesystems Dumped 11 3 8 (1:8)
Avg Dump Rate (k/s) 1241.5 1481.5 675.2
Tape Time (hrs:min) 2:22 2:08 0:15
Tape Size (meg) 3559.3 3106.1 453.2
Tape Used (%) 116.1 101.3 14.8 (level:#disks ...)
Filesystems Taped 10 2 8 (1:8)
Avg Tp Write Rate (k/s) 426.5 415.0 526.9
USAGE BY TAPE:
Label Time Size % Nb
FullBacks2 2:22 3559.3 116.1 10
FAILED AND STRANGE DUMP DETAILS:
/-- host01 /var lev 1 STRANGE
sendbackup: start [host01:/var level 1]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/tar -f... -
sendbackup: info end
? gtar: ./lib/amanda/FullBacks/index/host03/_/20040216_1.gz.tmp: Warning: Cannot stat:
No such file or directory
? gtar: ./lib/amanda/FullBacks/index/host02/_etc/20040216_1.gz.tmp: Warning: Cannot
stat: No such file or directory
| Total bytes written: 165437440 (158MB, 1.4MB/s)
sendbackup: size 161560
sendbackup: end
\--------
NOTES:
taper: tape FullBacks2 kb 3800896 fm 11 writing file: No space left on device
driver: going into degraded mode because of tape error.
DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s
-------------------------- --------------------------------- ------------
host03 / 1 110 128 -- 0:07 15.3 0:00 272.5
host03 /usr 1 12460 12480 -- 0:14 881.0 0:17 714.4
host03 /var 1 6530 6560 -- 0:09 763.8 0:10 632.0
host01 / 1 104910 104928 -- 1:341118.8 5:25 323.0
host01 /home 0 27489702748992 -- N/A N/A 118:08 387.9
host01 /home 1 148250 148256 -- 6:51 360.8 4:13 586.0
host01 /space/ICD 0 431650 431680 -- 2:003608.7 9:36 749.0
host01 /usr/local 0 11482601148260 -- 8:442189.7 N/A N/A
host01 /var 1 161560 161568 -- 1:521438.3 3:53 694.3
host02 /etc 1 2260 2272 -- 0:04 606.8 0:09 259.4
host02 /home 0 822370 822370 -- 16:17 841.3 FAILED ----
host02 /var 1 27820 27840 -- 0:37 761.7 0:33 842.5
(brought to you by Amanda version 2.4.4)
The dumps were flushed to tape FullBacks4.
The next tape Amanda expects to use is: FullBacks1.
STATISTICS:
Total Full Daily
-------- -------- --------
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 1:18
Dump Time (hrs:min) 0:00 0:00 0:00
Output Size (meg) 0.0 0.0 0.0
Original Size (meg) 0.0 0.0 0.0
Avg Compressed Size (%) -- -- --
Filesystems Dumped 0 0 0
Avg Dump Rate (k/s) -- -- --
Tape Time (hrs:min) 1:17 1:17 0:00
Tape Size (meg) 1924.5 1924.5 0.0
Tape Used (%) 62.8 62.8 0.0
Filesystems Taped 2 2 0
Avg Tp Write Rate (k/s) 426.8 426.8 --
USAGE BY TAPE:
Label Time Size % Nb
FullBacks4 1:17 1924.5 62.8 2
NOTES:
taper: tape FullBacks4 kb 1970752 fm 2 [OK]
DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s
-------------------------- --------------------------------- ------------
host03 / NO FILE TO FLUSH -----------------------------
host03 /usr NO FILE TO FLUSH -----------------------------
host03 /var NO FILE TO FLUSH -----------------------------
host01 / NO FILE TO FLUSH -----------------------------
host01 /home NO FILE TO FLUSH -----------------------------
host01 /space/ICD NO FILE TO FLUSH -----------------------------
host01 /usr/local 0 11482601148288 -- N/A N/A 34:02 562.4
host01 /var NO FILE TO FLUSH -----------------------------
host02 /etc NO FILE TO FLUSH -----------------------------
host02 /home 0 822370 822400 -- N/A N/A 42:56 319.2
host02 /var NO FILE TO FLUSH -----------------------------
(brought to you by Amanda version 2.4.4)
