we are having a problem with large( +2 gig) .Z compressed tarballs
(made w/ solaris compress utility) getting truncated/corrupted when
they are backed up w/ amanda (using gnutar).
Conveniently found this out when we attempted a restore.
What happens is that if one of these large compressed files is
encountered by the amanda backup, it silently stops backing up at that
point, and if the file is restored, it is a truncated/corrupted
version of the original file. gnutar has no problem backing
up/restoring larger (3.9 g) tarballs which are not compressed.
we will attempt a workaround by using a new version of gzip, but I am
curious as to whether this has been seen before, and whether it is a
gnutar bug, and if so , where I shoud report it. I am currently
running a test of gnutar w/o amanda on this file, but if it works
properly I am left scratching my head as to whether the problem is with
amanda or gnutar.
details follow:
--
amanda version Amanda-2.4.2p2
gnutar version 1.13.19
OS: solaris 2.6 /sparc
I have been testing on holding disk w/o using tape drive at all.
sample holding disk file :
# head foo.someuniversity.edu._raid.1
AMANDA: FILE 20011025 foo.someuniversity.edu /raid lev 1 comp N
program /usr/local/bin/gnutar
To restore, position tape at start of file and run:
dd if=<tape> bs=32k skip=1 | /usr/local/bin/gnutar -f... -
--
"restore" file off of holding disk:
# amrestore foo.someuniversity.edu._raid.1
amrestore: 0: restoring foo.someuniversity.edu._raid.20011025.1
# ls
-rw------- 1 amanda users 178585600 Oct 25 01:35
foo.someuniversity.edu._raid.1
-rw-r----- 1 user other 178552832 Oct 25 11:19
foo.someuniversity.edu._raid.20011025.1
# gnutar -tvf foo.someuniversity.edu._raid.20011025.1 > tableofcont
gnutar: Unexpected EOF in archive
gnutar: Error is not recoverable: exiting now
Note that the .Z file is the last file listed, even tho there are many
more to be backed up... obviously gnutar/amanda choked.
# tail -3 tableofcont
drwxr-xr-x www/nobody 29 2001-05-11 14:40:53
./tmp/etc/WebEvent/cals/cal7/
-rw-r--r-- user/other 0 2001-10-25 00:03:48 ./foo
-rw-r--r-- user/other 2111100973 2001-10-24 22:52:24
./backup/10-24-2001.tar.Z
--
# gnutar -xvpf foo.someuniversity.edu._raid.20011025.1
./backup/10-24-2001.tar.Z
./backup/10-24-2001.tar.Z
gnutar: Unexpected EOF in archive
gnutar: Unexpected EOF in archive
gnutar: Error is not recoverable: exiting now
# cd backup
# ls
total 344100
drwxr-xr-x 2 user other 512 Oct 25 11:38 ./
drwx------ 3 amanda users 512 Oct 25 11:38 ../
-rw-r--r-- 1 user other 176077824 Oct 25 11:39
10-24-2001.tar.Z
What the #%^#$%&*() is this file note much smaller size than the
original,
# file 10-24-2001.tar.Z
10-24-2001.tar.Z: USTAR tar archive
(???) (but this was a .Z compressed file previously ?
and just for grins, lets try to extract...)
#
# tar -tvf 10-24-2001.tar.Z
-rw-r--r-- 0/1 15704064 Oct 24 23:45 2001
07365732347/./backup/test.tar
-rw------- 1311/100 4715 Oct 24 12:41 2001
07365613346/./export/systems/deny
ce/.history
-rw------- 101/100 925207 Oct 24 15:43 2001
07321545437/./export/systems/kaa/mail/foo
tar: directory checksum error
curiouser and curiouser... none of these files were in the original
tar.Z file, in fact, looking at the numbers on the front, these files
were backed up by gnu tar, not by the solaris tar which created the orig
file.
---
amadmin DailySet info foo /raid
Current info for foo.someuniversity.edu /raid:
Stats: dump rates (kps), Full: 4142.0, 3329.0, 3367.0
Incremental: 3229.0, 3999.0, 2880.0
compressed size, Full: -100.0%,-100.0%,-100.0%
Incremental: -100.0%,-100.0%,-100.0%
Dumps: lev datestmp tape file origK compK secs
0 20011019 DailySet10 9 5103000 5103008 1232
1 20011025 0 174350 174368 54
---------------
amadmin DailySet version
build: VERSION="Amanda-2.4.2p2"
BUILT_DATE="Fri Jun 29 17:13:55 MST 2001"
BUILT_MACH="SunOS foo.someuniversity.edu 5.6 Generic_105181-11
sun4u sparc
SUNW,Ultra-5_10"
CC="gcc"
paths: bindir="/usr/local/bin" sbindir="/usr/local/sbin"
libexecdir="/usr/local/libexec" mandir="/usr/local/man"
AMANDA_TMPDIR="/tmp/amanda" AMANDA_DBGDIR="/tmp/amanda"
CONFIG_DIR="/usr/local/etc/amanda" DEV_PREFIX="/dev/dsk/"
RDEV_PREFIX="/dev/rdsk/" DUMP="/usr/sbin/ufsdump"
RESTORE="/usr/sbin/ufsrestore"
GNUTAR="/usr/local/bin/gnutar"
COMPRESS_PATH="/usr/local/bin/gzip"
UNCOMPRESS_PATH="/usr/local/bin/gzip"
MAILER="/usr/ucb/Mail"
listed_incr_dir="/usr/local/var/amanda/gnutar-lists"
defs: DEFAULT_SERVER="foo.someuniversity.edu"
DEFAULT_CONFIG="DailySet"
DEFAULT_TAPE_SERVER="foo.someuniversity.edu"
DEFAULT_TAPE_DEVICE="/dev/rmt/0bn" HAVE_MMAP HAVE_SYSVSHM
LOCKING=POSIX_FCNTL SETPGRP_VOID DEBUG_CODE
AMANDA_DEBUG_DAYS=4 BSD_SECURITY USE_AMANDAHOSTS
CLIENT_LOGIN="amanda" FORCE_USERID HAVE_GZIP
COMPRESS_SUFFIX=".gz" COMPRESS_FAST_OPT="--fast"
COMPRESS_BEST_OPT="--best" UNCOMPRESS_OPT="-dc"
--
gnutar --version
tar (GNU tar) 1.13.19
-
the original file:
# file 10-24-2001.tar.Z
10-24-2001.tar.Z: compressed data block compressed 16 bits
If anyone has a clue to give, I would appreciate it.
