My backup failed tonight as i forgot to turn on my tape drive before i went to bed. Nothing was written to my dump drive as it was too full from a previous aborted backup i had done. When i woke up i turned on the tape drive and ran the backup again as i have done before when this has happend, most DLEs ran fine but the one DLE i have with 'holdingdisk required' failed as there was too little space left on the holding disk.
I cleared out the holding disk and ran just that one failed DLE but it fails instantly and i get: FAILURE DUMP SUMMARY: planner: FATAL find_est_for_dp return NULL localhost storage RESULTS MISSING I tried running 'amcleanup -rv backup' but it did not do anything: # sudo -u amanda amcleanup -rv backup amcleanup: pid 2972312 is done amcleanup: pid 2972313 is done amcleanup: pid 2972311 is done amcleanup: pid 2972309 is done amcleanup: pid 2972305 is done amcleanup: no unprocessed logfile to clean up I also tested removing 'holdingdisk required' from the affected DLE but same result. Any ideas about what i should try? The logs don't seem to hold any more information (partial log from a RESULTS MISSING run): SENDING FLUSHES... Cleaning up holding disk '/dumps/' driver: adding holding disk 0 dir /dumps/ size 1949288448 chunksize 9007199254740896 ENDFLUSH SETTING UP FOR ESTIMATES... reserving 1949288448 out of 1949288448 for degraded-mode dumps driver: taper taper0 storage backup-storage tape_size 1499463680 driver: started dumper0 pid 2972312 driver: send-cmd time 0.001 to dumper0: START 20220416185052 planner: time 0.000: setting up estimates for localhost:storage driver: started dumper1 pid 2972313 driver: send-cmd time 0.001 to dumper1: START 20220416185052 setup_estimate: localhost:storage: command 0, options: none last_level 3 next_level0 14 level_days 1 getting estimates 0 (-3) 3 (-3) -1 (-3) planner: time 0.001: setting up estimates took 0.000 secs GETTING ESTIMATES... planner: find_est_for_dp return NULL driver: send-cmd time 0.001 to taper0: START-TAPER taper0 worker0-0 backup-storage 20220416185052 more about starting dumpers which never get used here until quitting. -Mail report 03:14 with drive off- Hostname: minerva Org : exuvo Config : backup Date : april 16, 2022 *** A TAPE ERROR OCCURRED: ['/dev/tape/by-id/scsi-DEC8320699' not found]. Some dumps may have been left in the holding disk. The next 2 tapes Amanda expects to use are: ROT09, ROT07. FAILURE DUMP SUMMARY: localhost efi lev 0 FAILED [can't dump 'holdingdisk never' dle in degraded mode] localhost boot lev 0 FAILED [can't dump 'holdingdisk never' dle in degraded mode] localhost root lev 1 FAILED [can't dump 'holdingdisk never' dle in degraded mode] localhost storage lev 3 FAILED [can't dump required holdingdisk when no holdingdisk space available ] localhost backup-minerva lev 0 FAILED [can't dump 'holdingdisk never' dle in degraded mode] localhost exuvo-desktop-root lev 1 FAILED [can't dump 'holdingdisk never' dle in degraded mode] localhost backup lev 0 FAILED [can't dump 'holdingdisk never' dle in degraded mode] STATISTICS: Total Full Incr. Level:# -------- -------- -------- -------- Estimate Time (hrs:min) 0:15 Run Time (hrs:min) 0:15 Dump Time (hrs:min) 0:00 0:00 0:00 Output Size (meg) 0.0 0.0 0.0 Original Size (meg) 0.0 0.0 0.0 Avg Compressed Size (%) -- -- -- DLEs Dumped 0 0 0 Avg Dump Rate (k/s) -- -- -- Tape Time (hrs:min) 0:00 0:00 0:00 Tape Size (meg) 0.0 0.0 0.0 Tape Used (%) 0.0 0.0 0.0 DLEs Taped 0 0 0 Parts Taped 0 0 0 Avg Tp Write Rate (k/s) -- -- -- NOTES: planner: Last full dump of localhost:efi on tape ROT05 overwritten in 4 runs. planner: Last full dump of localhost:boot on tape ROT05 overwritten in 4 runs. planner: Last full dump of localhost:root on tape ROT05 overwritten in 4 runs. planner: Last full dump of localhost:storage on tape ROT04 overwritten in 3 runs. planner: Last full dump of localhost:backup-minerva on tape ROT03 overwritten in 3 runs. planner: Last full dump of localhost:exuvo-desktop-root on tape ROT08 overwritten in 2 runs. planner: Last full dump of localhost:backup on tape ROT03 overwritten in 3 runs. planner: Full dump of localhost:backup-minerva promoted from 14 days ahead. planner: Full dump of localhost:boot promoted from 22 days ahead. planner: Full dump of localhost:efi promoted from 22 days ahead. planner: Full dump of localhost:backup promoted from 56 days ahead. DUMP SUMMARY: DUMPER STATS TAPER STATS HOSTNAME DISK L ORIG-GB OUT-GB COMP% MMM:SS KB/s MMM:SS KB/s --------------------------------- ---------------------- -------------- ------------- localhost backup FAILED localhost backup-minerva FAILED localhost boot FAILED localhost efi FAILED localhost exuvo-desktop-root FAILED localhost root FAILED localhost storage FAILED (brought to you by Amanda version 3.5.1) -Mail report 07:08 with drive on but dump disk still full- Hostname: minerva Org : exuvo Config : backup Date : april 16, 2022 These dumps were to tape ROT06. The next 2 tapes Amanda expects to use are: ROT09, ROT07. FAILURE DUMP SUMMARY: localhost storage lev 3 FAILED [can't dump required holdingdisk when no holdingdisk space available ] STRANGE DUMP SUMMARY: localhost root lev 1 STRANGE (see below) STATISTICS: Total Full Incr. Level:# -------- -------- -------- -------- Estimate Time (hrs:min) 0:21 Run Time (hrs:min) 1:40 Dump Time (hrs:min) 1:16 0:30 0:46 Output Size (meg) 245984.8 212008.5 33976.4 Original Size (meg) 274022.7 212399.5 61623.2 Avg Compressed Size (%) 89.8 99.8 55.1 DLEs Dumped 6 4 2 1:2 Avg Dump Rate (k/s) 55158.3 121007 12548.6 Tape Time (hrs:min) 1:00 0:31 0:29 Tape Size (meg) 245984.8 212008.5 33976.4 Tape Used (%) 16.8 14.5 2.3 DLEs Taped 6 4 2 1:2 Parts Taped 6 4 2 1:2 Avg Tp Write Rate (k/s) 69736.6 115600 20064.5 USAGE BY TAPE: Label Time Size % DLEs Parts ROT06 1:00 240G 16.8 6 6 STRANGE DUMP DETAILS: /-- localhost root lev 1 STRANGE sendbackup: info BACKUP=APPLICATION sendbackup: info APPLICATION=amgtar sendbackup: info RECOVER_CMD=/usr/lib/amanda/application/amgtar restore [./file-to-restore]+ sendbackup: info COMPRESS_SUFFIX=.gz sendbackup: info end | /usr/bin/tar: ./yuki: directory is on a different filesystem; not dumped | /usr/bin/tar: ./var/lib/samba/private/msg.sock/1871547: socket ignored | /usr/bin/tar: ./var/lib/samba/private/msg.sock/791703: socket ignored | Total bytes written: 3872423936 (3.7GiB, 53MiB/s) sendbackup: size 3781664 sendbackup: native-CRC a2e76bc7:3872423936 sendbackup: client-CRC a2e76bc7:3872423936 sendbackup: end \-------- NOTES: planner: Last full dump of localhost:efi on tape ROT05 overwritten in 4 runs. planner: Last full dump of localhost:boot on tape ROT05 overwritten in 4 runs. planner: Last full dump of localhost:root on tape ROT05 overwritten in 4 runs. planner: Last full dump of localhost:storage on tape ROT04 overwritten in 3 runs. planner: Last full dump of localhost:backup-minerva on tape ROT03 overwritten in 3 runs. planner: Last full dump of localhost:exuvo-desktop-root on tape ROT08 overwritten in 2 runs. planner: Last full dump of localhost:backup on tape ROT03 overwritten in 3 runs. planner: Full dump of localhost:backup-minerva promoted from 14 days ahead. planner: Full dump of localhost:boot promoted from 21 days ahead. planner: Full dump of localhost:efi promoted from 21 days ahead. planner: Full dump of localhost:backup promoted from 56 days ahead. taper: Slot 8 with label ROT06 is usable taper: tape ROT06 kb 251888474 fm 6 [OK] DUMP SUMMARY: DUMPER STATS TAPER STATS HOSTNAME DISK L ORIG-GB OUT-GB COMP% MMM:SS KB/s MMM:SS KB/s --------------------------------- ---------------------- ---------------- --------------- localhost backup 0 202 202 -- 28:39 123454.5 29:16 120829.2 localhost backup-minerva 0 5 4 92.1 1:14 68764.7 1:49 43280.9 localhost boot 0 0 0 -- 0:01 276524.3 0:10 18401.2 localhost efi 0 0 0 93.0 0:00 60278.6 0:03 6352.0 localhost exuvo-desktop-root 1 57 31 54.4 44:53 22026.9 27:46 19364.3 localhost root 1 4 2 66.9 1:19 47581.5 1:08 37218.9 localhost storage FAILED (brought to you by Amanda version 3.5.1) -Mail report 18:33 with drive on and empty dump drive, non-failing DLEs commented out in disklist- Hostname: minerva Org : exuvo Config : backup Date : april 16, 2022 The next 2 tapes Amanda expects to use are: ROT09, ROT07. FAILURE DUMP SUMMARY: planner: FATAL find_est_for_dp return NULL localhost storage RESULTS MISSING STATISTICS: Total Full Incr. Level:# -------- -------- -------- -------- Estimate Time (hrs:min) 0:00 Run Time (hrs:min) 0:02 Dump Time (hrs:min) 0:00 0:00 0:00 Output Size (meg) 0.0 0.0 0.0 Original Size (meg) 0.0 0.0 0.0 Avg Compressed Size (%) -- -- -- DLEs Dumped 0 0 0 Avg Dump Rate (k/s) -- -- -- Tape Time (hrs:min) 0:00 0:00 0:00 Tape Size (meg) 0.0 0.0 0.0 Tape Used (%) 0.0 0.0 0.0 DLEs Taped 0 0 0 Parts Taped 0 0 0 Avg Tp Write Rate (k/s) -- -- -- NOTES: planner: Last full dump of localhost:storage on tape ROT04 overwritten in 2 runs. driver: WARNING: got empty schedule from planner taper: Slot 4 with label ROT07 is usable DUMP SUMMARY: DUMPER STATS TAPER STATS HOSTNAME DISK L ORIG-GB OUT-GB COMP% MMM:SS KB/s MMM:SS KB/s -------------------------- ---------------------- -------------- ------------- localhost storage MISSING ------------------------------------------- (brought to you by Amanda version 3.5.1) -- Anton "exuvo" Olsson ex...@exuvo.se
