>Date: Tue, 19 Dec 2000 10:25:10 +0100
>From: Vincenzo Marchese <[EMAIL PROTECTED]>
>> I upgraded to 2.4.2 and I noticed that if dump won't fit on one disk that
>> amdump will go into endless loop trying to do a backup.
>I have the same problem, (sorry this is not a solution :) )
>amdump loops infinitely trying to dump the too-big-fs until you
>kill it by hand.
I've not seen that particular problem (though I use neither tar (for the
backup program) nor Linux, and my aggregate holding disk is about 38
GB).
But I saw something during the backups from this morning that may be
related.
I have 3 holding disks (snapshot while amdump is running):
eng-backup[1]% df -k /hd/tmp{0,1,2}
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/da2s1e 8890016 8178800 16 100% /hd/tmp0
/dev/da3s1e 17704608 2674256 13613984 16% /hd/tmp1
/dev/da4s1e 17669856 3354832 12901440 21% /hd/tmp2
But I had a couple of failures this morning:
eng-backup# amstatus Eng --failed --summ
Using /var/log/amanda/Eng/amdump from Tue Dec 19 01:53:00 PST 2000
shrimp:/shrimp/usr/local 0 [write_tapeheader: No space left on device] (6:07:45)
shrimp:/usr 0 [write_tapeheader: No space left on device] (6:08:01)
SUMMARY part real estimated
size size
partition : 149
estimated : 149 25019986k
failed : 2 1472051k ( 5.88%)
wait for dumping: 0 0k ( 0.00%)
dumping to tape : 0 0k ( 0.00%)
dumping : 2 2377536k 4662441k ( 50.99%) ( 9.50%)
dumped : 145 19893760k 18885494k (105.34%) ( 79.51%)
wait for writing: 0 0k 0k ( 0.00%) ( 0.00%)
writing to tape : 1 11437344k 11437820k (100.00%) ( 45.71%)
failed to tape : 0 0k 0k ( 0.00%) ( 0.00%)
taped : 144 8456416k 7447674k (113.54%) ( 33.80%)
98 dumpers idle : not-idle
taper writing, tapeq: 0
network free kps: 1180723
holding space : 24621960k ( 60.46%)
eng-backup#
So the aggregate space required by the 2 failed dumps would have been
around 1.5 GB; I had >25 GB free, but for reasons that aren't (yet)
clear to me, amanda tried to put the (first chunks of the) dump images
on a holding disk that didn't have the space. And there is no retrying
being done.
Looks as if I'm going to need to spend some time going over the logs on
this, to try to find out why amanda's notion of the amount of space on
/hd/tmp0 was so divergent from the OS's. :-( [I'll copy an extract of
the amdump log after my .sig just in case someone else who has more
familiarity with the logs can parse them and point out the weirdness before
I do; the holding disks are named hd0, hd1, & hd2, and refer to /hd/tmp0,
/hd/tmp1, & /hd/tmp2, respectively. The "df" output was done shortly
before I started writing this note, not when the failure occurred, so
there are undoubtedly *some* discrepancies, but the df output should still
be of some relevance. Also, I have the chunksize set to 1900 MB on each
holding disk -- less than 2 GB because although everything else works OK,
the numbers reported by Perl's "-s" operator go screwy when the high-
order bit flips; looks as if Perl's configure ends up using a 32-bit
quantity for the off_t for file size (result from stat()), which should
be an integral 64-bit quantity. (I've gone so far as to cobble up some
C & Perl code to verify the nature of the failure, but haven't yet found
where it's happening, so I circumvented the problem for now.)]
(This is the first time I've seen this behavior with amanda. I won't
object if it's the last....)
Cheers,
david (not the David who started the thread)
--
David Wolfskill [EMAIL PROTECTED] UNIX System Administrator
Desk: 650/577-7158 TIE: 8/499-7158 Cell: 650/759-0823
driver: send-cmd time 15155.688 to dumper0: CONTINUE
/hd/tmp1/20001219/shrimp._shrimp_home.0 1945600 15712
driver: state time 15155.692 free kps: 1180069 space: 23077320 taper: idle
idle-dumpers: 95 qlen tapeq: 0 runq: 2 roomq: 0 wakeup: 86400 driver-idle:
client-constrained
driver: interface-state time 15155.692 if : free 74149 if DE0: free 81920 if LOCAL:
free 1024000
driver: hdisk-state time 15155.692 hdisk 0: free 10377128 dumpers 3 hdisk 1: free
12700096 dumpers 2 hdisk 2: free 96 dumpers 0
driver: result time 15270.198 from dumper0: RQ-MORE-DISK 01-00289
driver: send-cmd time 15270.198 to dumper0: CONTINUE
/hd/tmp1/20001219/shrimp._shrimp_home.0 1945600 16480
driver: state time 15270.201 free kps: 1180069 space: 23060840 taper: idle
idle-dumpers: 95 qlen tapeq: 0 runq: 2 roomq: 0 wakeup: 86400 driver-idle:
client-constrained
driver: interface-state time 15270.201 if : free 74149 if DE0: free 81920 if LOCAL:
free 1024000
driver: hdisk-state time 15270.201 hdisk 0: free 10377128 dumpers 3 hdisk 1: free
12683616 dumpers 2 hdisk 2: free 96 dumpers 0
driver: result time 15284.895 from dumper0: DONE 01-00289 752051 331200 2621 [sec
2621.111 kb 331200 kps 126.4 orig-kb 752051]
driver: finished-cmd time 15285.121 dumper0 dumped shrimp:/shrimp/home
driver: send-cmd time 15285.121 to taper: FILE-WRITE 00-00290
/hd/tmp1/20001219/shrimp._shrimp_home.0 shrimp /shrimp/home 0 20001219
driver: send-cmd time 15285.124 to dumper0: FILE-DUMP 01-00291
/hd/tmp0/20001219/shrimp._shrimp_usr_local.0 shrimp /shrimp/usr/local 0 1970:1:1:0:0:0
1945600 DUMP 96 |;bsd-auth;index;
driver: state time 15285.126 free kps: 1180002 space: 22011688 taper: writing
idle-dumpers: 95 qlen tapeq: 0 runq: 1 roomq: 0
wakeup: 15 driver-idle: start-wait
driver: interface-state time 15285.126 if : free 74082 if DE0: free 81920 if LOCAL:
free 1024000
driver: hdisk-state time 15285.126 hdisk 0: free 10377128 dumpers 3 hdisk 1: free
11634560 dumpers 1 hdisk 2: free 0 dumpers 1
driver: result time 15285.929 from dumper0: FAILED 01-00291 [write_tapeheader: No
space left on device]
dumper: kill index command
driver: send-cmd time 15300.947 to dumper0: FILE-DUMP 01-00292
/hd/tmp0/20001219/shrimp._usr.0 shrimp /usr 0 1970:1:1:0:0:0 1945600 DUMP 96
|;bsd-auth;index;
driver: state time 15300.951 free kps: 1180109 space: 22665736 taper: writing
idle-dumpers: 95 qlen tapeq: 0 runq: 0 roomq: 0
wakeup: 86400 driver-idle: not-idle
driver: interface-state time 15300.951 if : free 74189 if DE0: free 81920 if LOCAL:
free 1024000
driver: hdisk-state time 15300.951 hdisk 0: free 10377128 dumpers 3 hdisk 1: free
12288608 dumpers 1 hdisk 2: free 0 dumpers 1
driver: result time 15301.273 from dumper0: FAILED 01-00292 [write_tapeheader: No
space left on device]
dumper: kill index command
driver: state time 15316.285 free kps: 1180161 space: 23074856 taper: writing
idle-dumpers: 96 qlen tapeq: 0 runq: 0 roomq: 0
wakeup: 86400 driver-idle: not-idle
driver: interface-state time 15316.285 if : free 74241 if DE0: free 81920 if LOCAL:
free 1024000
driver: hdisk-state time 15316.285 hdisk 0: free 10377128 dumpers 3 hdisk 1: free
12697632 dumpers 1 hdisk 2: free 96 dumpers 0
taper: reader-side: got label E000032 filenum 143
driver: result time 15349.366 from taper: DONE 00-00290 E000032 143 [sec 64.241 kb
331232 kps 5156.0 {wr: writers 10351 rdwait 0.000 wrwait 42.842 filemark 2.229}]
driver: finished-cmd time 15349.454 taper wrote shrimp:/shrimp/home
driver: state time 15349.455 free kps: 1180161 space: 23406088 taper: idle
idle-dumpers: 96 qlen tapeq: 0 runq: 0 roomq: 0 wakeup: 86400 driver-idle: not-idle
driver: interface-state time 15349.455 if : free 74241 if DE0: free 81920 if LOCAL:
free 1024000
driver: hdisk-state time 15349.455 hdisk 0: free 10377128 dumpers 3 hdisk 1: free
13028864 dumpers 1 hdisk 2: free 96 dumpers 0
david
--
David Wolfskill [EMAIL PROTECTED] UNIX System Administrator
Desk: 650/577-7158 TIE: 8/499-7158 Cell: 650/759-0823
