Yesterdays 3411 failed last night, all DEL's timed out waiting for writing
to tape.
The install, which ends in an amcheck:
install done, now running amcheck as the user amanda
Amanda Tape Server Host Check
-----------------------------
Holding disk /dumps: 21712896 kB disk space available, using 21200896 kB
Searching for label 'Dailys-20':found in slot 20: volume 'Dailys-20'
Will write to volume 'Dailys-20' in slot 20.
NOTE: skipping tape-writable test
Server check took 0.312 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 2 hosts checked in 6.333 seconds. 0 problems found.
(brought to you by Amanda 3.2.0alpha.svn.3411)
==========
but no mail or printout was received, and my 32 count active disklist comes
back as 32 lines of this:
coyote:/GenesAmandaHelper-0.6 2 141545k dump done (2:01:42), process
terminated while waiting for writing to tape
coyote:/bin 0 11520k dump done (3:42:09), process
terminated while waiting for writing to tape
coyote:/boot 0 131910k dump done (1:54:59), process
terminated while waiting for writing to tape
coyote:/etc 0 6660k dump done (3:42:45), process
terminated while waiting for writing to tape
coyote:/home 0 12775661k dump done (3:33:36), process
terminated while waiting for writing to tape
coyote:/lib 1 3222k dump done (3:44:36), process
terminated while waiting for writing to tape
coyote:/opt 1 356k dump done (3:46:23), process
terminated while waiting for writing to tape
coyote:/root 0 4893k dump done (3:41:55), process
terminated while waiting for writing to tape
coyote:/sbin 0 21850k dump done (1:55:25), process
terminated while waiting for writing to tape
coyote:/tmp 1 3819k dump done (3:33:39), process
terminated while waiting for writing to tape
coyote:/usr/X11R6 1 1k dump done (3:42:24), process
terminated while waiting for writing to tape
coyote:/usr/bin 1 160490k dump done (3:43:46), process
terminated while waiting for writing to tape
coyote:/usr/dlds/misc 1 40k dump done (3:44:09), process
terminated while waiting for writing to tape
coyote:/usr/dlds/rpms 0 273580k dump done (1:54:53), process
terminated while waiting for writing to tape
coyote:/usr/dlds/tgzs 1 10k dump done (3:48:00), process
terminated while waiting for writing to tape
coyote:/usr/include 0 4349k dump done (3:42:57), process
terminated while waiting for writing to tape
coyote:/usr/lib 0 666070k dump done (3:41:53), process
terminated while waiting for writing to tape
coyote:/usr/libexec 1 4k dump done (3:48:15), process
terminated while waiting for writing to tape
coyote:/usr/local 1 19114k dump done (3:43:22), process
terminated while waiting for writing to tape
coyote:/usr/movies 1 40k dump done (3:43:54), process
terminated while waiting for writing to tape
coyote:/usr/music 0 103340k dump done (1:55:13), process
terminated while waiting for writing to tape
coyote:/usr/pix 1 140k dump done (3:43:25), process
terminated while waiting for writing to tape
coyote:/usr/sbin 0 20424k dump done (3:34:04), process
terminated while waiting for writing to tape
coyote:/usr/share 1 153125k dump done (3:46:09), process
terminated while waiting for writing to tape
coyote:/usr/src 1 4233k dump done (3:48:00), process
terminated while waiting for writing to tape
coyote:/var 0 3187338k dump done (2:01:18), process
terminated while waiting for writing to tape
shop:/etc 1 20k dump done (1:55:54), process
terminated while waiting for writing to tape
shop:/home 1 768k dump done (1:55:11), process
terminated while waiting for writing to tape
shop:/usr/lib/amanda 1 2k dump done (1:55:00), process
terminated while waiting for writing to tape
shop:/usr/local 1 1k dump done (1:55:30), process
terminated while waiting for writing to tape
shop:/usr/src 1 121k dump done (1:54:45), process
terminated while waiting for writing to tape
shop:/var/lib/amanda 1 115k dump done (1:55:40), process
terminated while waiting for writing to tape
And I have 18Gb sitting in /dumps.
pclos updated 352 packages last night before amanda was scheduled to run,
so I am going to reboot and try a flush, reporting back when I have done
that.
Continued, a flush didn't. Both /tmp/amanda-dbg/Daily/server/taper-(date
hash) files are virtually empty:
Fri Sep 17 09:49:48 2010: taper: pid 29407 ruid 501 euid 501 version
3.2.0alpha.svn.3411: start at Fri Sep 17 09:49:48 2010
Fri Sep 17 09:49:48 2010: taper: pid 29407 ruid 501 euid 501 version
3.2.0alpha.svn.3411: rename at Fri Sep 17 09:49:48 2010
Fri Sep 17 09:49:48 2010: taper: Amanda::Taper::Scan::traditional stage 1:
search for oldest reusable volume
Fri Sep 17 09:49:48 2010: taper: Amanda::Taper::Scan::traditional oldest
reusable volume is 'Dailys-20'
Fri Sep 17 09:49:48 2010: taper: Amanda::Taper::Scan::traditional stage 1:
searching oldest reusable volume 'Dailys-20'
Fri Sep 17 09:49:48 2010: taper: Amanda::Taper::Scan::traditional result:
'Dailys-20' on file:/amandatapes/Dailys/drive0 slot 20, mode 2
Fri Sep 17 09:49:48 2010: taper: pid 29407 finish time Fri Sep 17 09:49:48
2010
I restored my /etc/aliases file before I ran the flush, and that got me 3
emails, one from my script and 2 from the failed amanda runs. The first
emailed report looks normal. But the report from the flush is really
strange:
Hostname: coyote.coyote.den
Org : The Coyote Den
Config : Daily
Date : September 17, 2010
There are 17694761k of dumps left in the holding disk.
They will be flushed on the next run.
The next tape Amanda expects to use is: Dailys-20.
STATISTICS:
Total Full Incr.
-------- -------- --------
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:00
Dump Time (hrs:min) 0:00 0:00 0:00
Output Size (meg) 0.0 0.0 0.0
Original Size (meg) 0.0 0.0 0.0
Avg Compressed Size (%) -- -- --
Filesystems Dumped 0 0 0
Avg Dump Rate (k/s) -- -- --
Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 0.0 0.0 0.0
Tape Used (%) 0.0 0.0 0.0
Filesystems Taped 0 0 0
Parts Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --
DUMP SUMMARY:
DUMPER STATS
TAPER STATS
HOSTNAM DISK L ORIG-kB OUT-kB COMP% MMM:SS KB/s
MMM:SS KB/s
------------------------------------- -----------------------------------
-------------
coyote /GenesAmandaHelper-0.6 NO FILE TO FLUSH
--------------------------------
coyote /bin NO FILE TO FLUSH
--------------------------------
coyote /boot NO FILE TO FLUSH
--------------------------------
coyote /etc NO FILE TO FLUSH
--------------------------------
coyote /home NO FILE TO FLUSH
--------------------------------
coyote /lib NO FILE TO FLUSH
--------------------------------
coyote /opt NO FILE TO FLUSH
--------------------------------
coyote /root NO FILE TO FLUSH
--------------------------------
coyote /sbin NO FILE TO FLUSH
--------------------------------
coyote /tmp NO FILE TO FLUSH
--------------------------------
coyote /usr/X11R6 NO FILE TO FLUSH
--------------------------------
coyote /usr/bin NO FILE TO FLUSH
--------------------------------
coyote /usr/dlds/misc NO FILE TO FLUSH
--------------------------------
coyote /usr/dlds/rpms NO FILE TO FLUSH
--------------------------------
coyote /usr/dlds/tgzs NO FILE TO FLUSH
--------------------------------
coyote /usr/include NO FILE TO FLUSH
--------------------------------
coyote /usr/lib NO FILE TO FLUSH
--------------------------------
coyote /usr/libexec NO FILE TO FLUSH
--------------------------------
coyote /usr/local NO FILE TO FLUSH
--------------------------------
coyote /usr/movies NO FILE TO FLUSH
--------------------------------
coyote /usr/music NO FILE TO FLUSH
--------------------------------
coyote /usr/pix NO FILE TO FLUSH
--------------------------------
coyote /usr/sbin NO FILE TO FLUSH
--------------------------------
coyote /usr/share NO FILE TO FLUSH
--------------------------------
coyote /usr/src NO FILE TO FLUSH
--------------------------------
coyote /var NO FILE TO FLUSH
--------------------------------
shop /etc NO FILE TO FLUSH
--------------------------------
shop /home NO FILE TO FLUSH
--------------------------------
shop /usr/lib/amanda NO FILE TO FLUSH
--------------------------------
shop /usr/local NO FILE TO FLUSH
--------------------------------
shop /usr/src NO FILE TO FLUSH
--------------------------------
shop /var/lib/amanda NO FILE TO FLUSH
--------------------------------
(brought to you by Amanda version 3.2.0alpha.svn.3411)
And there is still 17Gb of dumps in /dumps. Me goes off scratching head in
puzzlement, and to reinstall the previous version to see if it still works,
that was 3402 IIRC.
And that (3402) did not allow the flush either. Watching disk activity in
gkrellm, I see one access to /dev/sdc1 (/amandatapes, and it is mounted
according to mount) as the flush script starts, then about 20 seconds of
nothing, and my script exits. Without reporting an error! No mail. run
newaliases, didn't help.
>From the log my script makes:
/GenesAmandaHelper-0.6//flush.sh started on 17/09/10 10:22:19 with arg Daily
CONFIGNAME = Daily
First put the tapelist in order with a run of amcheck
running /usr/local/sbin/amcheck Daily
Amanda Tape Server Host Check
-----------------------------
Holding disk /dumps: 3997696 kB disk space available, using 3485696 kB
Searching for label 'Dailys-20':found in slot 20: volume 'Dailys-20'
Will write to volume 'Dailys-20' in slot 20. <--------------
NOTE: skipping tape-writable test
Server check took 0.182 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 2 hosts checked in 6.270 seconds. 0 problems found.
(brought to you by Amanda 3.2.0alpha.svn.3402)
Back in /GenesAmandaHelper-0.6//flush.sh - and the lists should be sane
line 72 /GenesAmandaHelper-0.6//flush.sh tapenum from
/usr/local/etc/amanda/Daily/chg-disk-slot is now=2 <-----------
TAPECYCLE=30
RUNTAPES=1
This script is being run as /GenesAmandaHelper-0.6//flush.sh
Something went wrong, the backup or flush was not done.
And here is a clue, note that amcheck says slot 20, but that my log says
slot 2.
And an ls -l /amandatapes/Dailys/
lrwxrwxrwx 1 amanda amanda 26 2010-08-29 16:35 data ->
/amandatapes/Dailys//slot2/
And note the date, that is several versions back up the log now. svn3343
TBE, so that is reinstalling now.
And at 3343, chg-disk-slot, nor the data link in /amandatapes/Dailys is
being updated, both are still at slot 2. 3341 is installing now. But its
interesting, every amcheck report says slot 20, and according to the dates
on the slots, 19 was the most recently used.
If 3341 doesn't work either, I'm headed for the sawbones to go see about a
sore throat. Either something that pclos upgraded yesterday is broken or
amanda, and amanda worked last before this last update late last night.
I'm voting for pclos at this point since that's the only way I can put 2 &
2 together & get 4.
--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
"Say yur prayers, yuh flea-pickin' varmint!"
-- Yosemite Sam
