Hello all-
I'm new to list, having subscribed myself today because I'm having problems
with an Amanda backup system I'm administering for a co-worker while he's on
vacation. Here's the situation:
The system has been running fine for about a month. On monday, we added a
new machine to the disks list. Since then, the backups keep failing. I
believe I've traced the problem to an error I found in the amdump files:
dumper: error [bad command after NO-ROOM: 5]
dumper: error [bad command after NO-ROOM: 5]
Unfortunately, the result of this is that Amanda dies with the holding disk
full. To further complicate matters, the holding disk specified is a
subdirectory on the /var partition which means (you guessed it) neither
Amanda nor indeed anything that needs to write to /var in order to function
can do so. Subsequently, not only does Amanda die, it dies in such a way
that it can't let anyone know.
Now, the strange parts. The amount of space alloted to the holding disk is
supposed to be 1290M, as dictated (I believe) by these lines in the
amanda.conf.main file:
holdingdisk hd1 {
comment "main holding disk"
directory "/var/tmp" # where the holding disk is
use 1290 Mb # how much space can we use on it
# a negative value mean:
# use all space except that value
chunksize -1 # size of chunk if you want big dump to be
# dumped on multiple files on holding disks
# N Kb/Mb/Gb split disks in chunks of size N
# 0 split disks in INT_MAX/1024 Kb chunks
# -1 same as -INT_MAX/1024 (see below)
# -N Kb/Mb/Gb dont split, dump larger
# filesystems directly to tape
# (example: -2 Gb)
# chunksize 2 Gb
}
However, I've seen Amanda use more than 1290Mb. In fact, it uses as much as
it can, which goes well over 1500Mb until it simply runs out of space (the
partition is only 1700Mb total, and usually has ~1500Mb available when
Amanda runs)
I've since commented out the disks that were added on Monday, and am running
a backup now; we'll see if it suceeds. My questions are:
1) Why is amanda ignoring the 1290Mb allocation?
2) Why is it dying with the NO-ROOM bad command error? (NOTE: I believe this
may be fixed with a patch of the server...is that true?)
3) Why is it dying at all? Out of space on the disk? I don't think the
tapes are filling up...
I'm thinking of using a -250M for the "use" directive in the conf file,
hoping maybe then it would pay attention. Also thinking of setting the
comprate to 100%; perhaps Amanda underestimated the new disks, and that's
why it failed? I'm going to attempt these things after the current forced
dump finishes. However IMHO they're work-arounds, not really solutions.
Let me know any opinions you've got. Below, you can find other snippets of
log & config files I thought might be helpful. I apologize if this is too
basic a question; I am not the regular administrator of this system, and
have had to learn amanda "on-the-fly" today.
Thanks in advance,
Brian
RELEVANT LOGS:
--------------
***************************************************************************
***Summary of the amdump log that was noted with the NO-ROOM error above***
***************************************************************************
SUMMARY part real estimated
size size
partition : 86
estimated : 86 14614544k
failed : 3 98230k
wait for dumping: 12 11734148k
dumping to tape : 0 0k
dumping : 3 0k 1552826k
dumped : 68 1232320k 1229340k
wait for writing: 0 0k 0k
writing to tape : 0 0k 0k
failed to tape : 0 0k 0k
taped : 68 1232320k 1229340k
1 dumpers idle : file-too-large
taper writing, tapeq: 0
network free kps: 13509
holding space : 43918
***************************************************************************
***Tail end of the log amdump file above
***
***************************************************************************
driver: state time 2572.384 free kps: 13509 space: 95590 taper: writing
idle-dumpers: 1 qlen tapeq: 0 runq: 12 stoppedq: 0 wakeup: 86400
driver-idle: file-too-large
driver: interface-state time 2572.384 if : free 8509 if ETH0: free 10000 if
LOCAL: free 10000
driver: hdisk-state time 2572.384 hdisk 0: free 43918 dumpers 1 hdisk 1:
free 51672 dumpers 2
driver: result time 2585.151 from dumper0: NO-ROOM 01-00139
driver: state time 2585.152 free kps: 13509 space: 95590 taper: writing
idle-dumpers: 1 qlen tapeq: 0 runq: 12 stoppedq: 1 wakeup: 86400
driver-idle: file-too-large
driver: interface-state time 2585.152 if : free 8509 if ETH0: free 10000 if
LOCAL: free 10000
driver: hdisk-state time 2585.152 hdisk 0: free 43918 dumpers 1 hdisk 1:
free 51672 dumpers 2
driver: result time 2585.198 from dumper3: NO-ROOM 03-00004
driver: state time 2585.198 free kps: 13509 space: 43918 taper: writing
idle-dumpers: 1 qlen tapeq: 0 runq: 12 stoppedq: 2 wakeup: 86400
driver-idle: file-too-large
driver: interface-state time 2585.198 if : free 8509 if ETH0: free 10000 if
LOCAL: free 10000
driver: hdisk-state time 2585.198 hdisk 0: free 43918 dumpers 1 hdisk 1:
free -567700 dumpers 2
taper: reader-side: got label <TAPENAME> filenum 68
driver: result time 2684.594 from taper: DONE 02-00142 <TAPENAME> 68 [sec
112.207 kb 184448 kps 1643.8 {wr: writers 5764 rdwait 17.054 wrwait 91.532
filemark 1.654}]
driver: infofile update failed (<COMPNAME>,/home)
dumper: error [bad command after NO-ROOM: 5]
dumper: error [bad command after NO-ROOM: 5]
gzip: stdout: No space left on device
amdump: end at Wed Apr 18 14:02:41 PDT 2001
gzip: stdout: No space left on device
dumper: log file write error: No space left on device
***************************************************************************
***Output of the amadmin version command
***
***************************************************************************
build: VERSION="Amanda-2.4.1p1"
BUILT_DATE="Thu Feb 10 15:53:50 EST 2000"
BUILT_MACH="Linux porky.devel.redhat.com 2.2.5-22smp #1 SMP Wed Jun 2
09:11:51 EDT 1999 i686 unknown"
CC="gcc"
paths: bindir="/usr/sbin" sbindir="/usr/sbin"
libexecdir="/usr/lib/amanda" mandir="/usr/man"
CONFIG_DIR="/etc/amanda" DEV_PREFIX="/dev/"
RDEV_PREFIX="/dev/r" SAMBA_CLIENT="/usr/bin/smbclient"
GNUTAR="/bin/tar" COMPRESS_PATH="/usr/bin/gzip"
UNCOMPRESS_PATH="/usr/bin/gzip" MAILER="/usr/bin/Mail"
listed_incr_dir="/var/lib/amanda/gnutar-lists"
defs: DEFAULT_SERVER="localhost" DEFAULT_CONFIG="<SETNAME>"
DEFAULT_TAPE_SERVER="localhost"
DEFAULT_TAPE_DEVICE="/dev/null" HAVE_MMAP HAVE_SYSVSHM
LOCKING=POSIX_FCNTL SETPGRP_VOID DEBUG_CODE BSD_SECURITY
USE_AMANDAHOSTS CLIENT_LOGIN="operator" FORCE_USERID
HAVE_GZIP COMPRESS_SUFFIX=".gz" COMPRESS_FAST_OPT="--fast"
COMPRESS_BEST_OPT="--best" UNCOMPRESS_OPT="-dc"
