Ok, so I ran amflush again. It flushed 2 of the 3 outstanding DLE's data to daily-1, but the email I received includes:
The dumps were flushed to tape daily-1. The next tape Amanda expects to use is: daily-2. FAILURE AND STRANGE DUMP SUMMARY: minerva /usr/local/clients lev 0 FAILED [input: Can't read data: : Input/output error] And the holding disk still contains a folder with Friday's date and a 30GB file for the DLE mentioned above. What on earth is going on?? Joe Donner wrote: > > Red Hat Enterprise 3 doesn't seem to have strace as a command. > > I thought rather than killing the processes manually, I'd reboot the > server and see if amcleanup runs as included in /etc/rc.d/rc.local > (thought I may as well test that). > > Now the server came back up, and none of the amanda services are active > anymore (unsurprisingly). Nothing seemed to happen, so I did a manual > amcleanup, with these results: > > amcleanup: no unprocessed logfile to clean up. > Scanning /mnt/hdb1... > 20060714: found Amanda directory. > > So I'm thinking that this backup run is now finally broken. > > Next I thought I'll run amflush and see what happens. It outputs this: > > Scanning /mnt/hdb1... > 20060714: found Amanda directory. > > Today is: 20060717 > Flushing dumps in 20060714 to tape drive "/dev/nst0". > Expecting tape daily-1 or a new tape. (The last dumps were to tape > daily-3) > Are you sure you want to do this [yN]? y > Running in background, you can log off now. > You'll get mail when amflush is finished. > > Now what I notice is that it asks for the tape called daily-1, whereas the > tape I used for Friday's backup was daily-3. Does this mean that daily-3 > was filled up and caused this whole issue? > > Which brings me to another question. I've used these tapes before for > testing. Will Amanda have appended Friday's backup to what was already on > the tape daily-3, or does it overwrite data previously written to that > tape each time a new backup runs? The reason I ask this is that the tape > drive capacity is 160GB, and I believe that I'm trying to back up a lot > less data than that. > > After I rebooted, I got this email from Amanda. As you can see, it only > used 4.7% of the tape: > > *** THE DUMPS DID NOT FINISH PROPERLY! > > These dumps were to tape daily-3. > The next tape Amanda expects to use is: daily-1. > > FAILURE AND STRANGE DUMP SUMMARY: > cerberus /.fonts.cache-1 lev 0 FAILED [disk /.fonts.cache-1 offline on > cerberus?] > cerberus /.autofsck lev 0 FAILED [disk /.autofsck offline on > cerberus?] > > > STATISTICS: > Total Full Daily > -------- -------- -------- > Estimate Time (hrs:min) 0:04 > Run Time (hrs:min) 0:16 > Dump Time (hrs:min) 3:07 3:07 0:00 > Output Size (meg) 56785.8 56785.8 0.0 > Original Size (meg) 136236.1 136236.1 0.0 > Avg Compressed Size (%) 41.7 41.7 -- > Filesystems Dumped 107 107 0 > Avg Dump Rate (k/s) 5169.8 5169.8 -- > > Tape Time (hrs:min) 0:13 0:13 0:00 > Tape Size (meg) 7259.3 7259.3 0.0 > Tape Used (%) 4.7 4.7 0.0 > Filesystems Taped 104 104 0 > Avg Tp Write Rate (k/s) 9801.6 9801.6 -- > > USAGE BY TAPE: > Label Time Size % Nb > daily-3 0:13 7259.3 4.7 104 > > And then, after I ran amflush, I got an email saying this (I didn't > actually put daily-1 into the drive): > > *** A TAPE ERROR OCCURRED: [cannot overwrite active tape daily-3]. > Some dumps may have been left in the holding disk. > Run amflush again to flush them to tape. > The next tape Amanda expects to use is: daily-1. > > And when I now do amstatus daily, I get: > > Using /var/lib/amanda/daily/amflush.1 from Mon Jul 17 12:58:42 BST 2006 > > minerva:/home 0 8774296k waiting to flush > minerva:/usr/local/clients 0 32253287k waiting to flush > minerva:/usr/local/development 0 9687648k waiting to flush > > I feel a headache coming on again... > > Any suggestions as how to best proceed? > > > > Paul Bijnens wrote: >> >> On 2006-07-17 13:32, Joe Donner (sent by Nabble.com) wrote: >>> and ps -fu amanda outputs: >>> >>> UID PID PPID C STIME TTY TIME CMD >>> amanda 2136 2135 0 Jul14 ? 00:00:00 /bin/sh /usr/sbin/amdump >>> daily >>> amanda 2145 2136 0 Jul14 ? 00:00:02 /usr/lib/amanda/driver >>> daily >>> amanda 2146 2145 0 Jul14 ? 00:00:52 taper daily >>> amanda 2147 2146 0 Jul14 ? 00:00:34 taper daily >>> amanda 2148 2145 0 Jul14 ? 00:12:55 dumper0 daily >>> amanda 2153 2145 0 Jul14 ? 00:00:19 dumper1 daily >>> amanda 2154 2145 0 Jul14 ? 00:00:00 dumper2 daily >>> amanda 2155 2145 0 Jul14 ? 00:00:00 dumper3 daily >>> >>> Does this tell anyone anything? >> >> It means the processes are still alive. >> >> Just a wild guess... Maybe you have specified a manual changer, and >> Amanda is just waiting for you to manually insert the next tape? >> >> Now find out what they are doing, and why it takes days to proceed. >> >> As root or amanda you can trace a process and see if it does somehting >> else, or is just sleeping on some event that will not happen: >> >> strace -p pid-of-the-process >> >> There are two taper processes, one reads from the holdingdisk file >> into a shared memory region, while the other one writes the bytes >> from shared memory to tape. When there is no holdingdisk file, then >> maybe the reader-taper is reading from a network socket? >> And maybe you specified a long dtimeout? >> >> >> -- >> Paul Bijnens, xplanation Technology Services Tel +32 16 397.511 >> Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM Fax +32 16 397.512 >> http://www.xplanation.com/ email: [EMAIL PROTECTED] >> *********************************************************************** >> * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * >> * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * >> * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * >> * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * >> * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * >> * ... "Are you sure?" ... YES ... Phew ... I'm out * >> *********************************************************************** >> >> >> > > -- View this message in context: http://www.nabble.com/Unravel-amstatus-output-tf1953587.html#a5360402 Sent from the Amanda - Users forum at Nabble.com.
