Re: amtapetype - silly thing I tried
Jon, amtapetype should terminate once it fill the disk. Jean-Louis Jon LaBadie wrote: Back to back posts to the list on amtapetype and the file:driver made me wonder, will amtapetype work with virtual tapes? Not that it would be of any benefit, just wondered if it would. Well, to my surprise, amtapetype does seem to work with vtapes. But ... It recognized the slot and amanda tape in the slot, refusing to go on. It could be forced to overwrite the amanda tape in the slot with the -f option. It did its compression test. It wrote its files to the virtual tape slot. And wrote and wrote and wrote. I.e. unlike amdump/amflush that appear to see a pseudo EOF when their writing reached the 14GB size listed in the tapetype definition (and the estimate, -e option), amtapetype never sees the EOF. Of course it is trying to figure out what that size limit is supposed to be, so maybe it is reasonable that it never got an EOF ;) Not sure which surprised me most, that it started and worked well, or that it did not end normally.
tape spanning problem with 2.5.2-20070523
Hi, Looks like tape spanning is failing. I can't recover the DLE and looking at the actual content of the tape: AMANDA: SPLIT_FILE 20070530 yorick /data/narsad/narsad1 part 1/2 lev 1 comp N program /usr/freeware/bin/tar AMANDA: SPLIT_FILE 20070530 yorick /data/narsad/narsad1 part 2/2 lev 1 comp N program /usr/freeware/bin/tar [...] # mt -f /hw/tape/tps21d1nrnsv fsf 1 # dd if=/hw/tape/tps21d1nrnsv bs=32k skip=1 | /usr/freeware/bin/tar -tf - [...] ./analysis/s17/04mm/kadulina_yara_20050727_105815_3_mri_MC.log ./analysis/s17/04mm/kadulina_yara_20050727_105815_3_mri_MC.mnc ./analysis/s17/04mm/kadulina_yara_20050727_105815_4_mri.mnc 327679+0 records in 327679+0 records out /usr/freeware/bin/tar: Unexpected EOF in archive /usr/freeware/bin/tar: Error is not recoverable: exiting now # mt -f /hw/tape/tps21d1nrnsv rewind # mt -f /hw/tape/tps21d1nrnsv fsf 2 # dd if=/hw/tape/tps21d1nrnsv bs=32k skip=1 | /usr/freeware/bin/tar -tf - /usr/freeware/bin/tar: This does not look like a tar archive /usr/freeware/bin/tar: Skipping to next header /usr/freeware/bin/tar: Archive contains obsolescent base-64 headers 253158+0 records in 253158+0 records out /usr/freeware/bin/tar: Error exit delayed from previous errors Any ideas? jf
Re: tape spanning problem with 2.5.2-20070523
You can't retrieve the split_file separately, you must concatenate them before untaring. You can use amfetchdump: amfetchdump -p -d /hw/tape/tps21d1nrnsv config yorick /data/narsad/narsad1 20070530 | /usr/freeware/bin/tar -tf - or: mt -f /hw/tape/tps21d1nrnsv rewind mt -f /hw/tape/tps21d1nrnsv fsf 1 dd if=/hw/tape/tps21d1nrnsv bs=32k skip=1 part1 dd if=/hw/tape/tps21d1nrnsv bs=32k skip=1 part2 cat part1 part2 | /usr/freeware/bin/tar -tf - Jean-Francois Malouin wrote: Hi, Looks like tape spanning is failing. I can't recover the DLE and looking at the actual content of the tape: AMANDA: SPLIT_FILE 20070530 yorick /data/narsad/narsad1 part 1/2 lev 1 comp N program /usr/freeware/bin/tar AMANDA: SPLIT_FILE 20070530 yorick /data/narsad/narsad1 part 2/2 lev 1 comp N program /usr/freeware/bin/tar [...] # mt -f /hw/tape/tps21d1nrnsv fsf 1 # dd if=/hw/tape/tps21d1nrnsv bs=32k skip=1 | /usr/freeware/bin/tar -tf - [...] ./analysis/s17/04mm/kadulina_yara_20050727_105815_3_mri_MC.log ./analysis/s17/04mm/kadulina_yara_20050727_105815_3_mri_MC.mnc ./analysis/s17/04mm/kadulina_yara_20050727_105815_4_mri.mnc 327679+0 records in 327679+0 records out /usr/freeware/bin/tar: Unexpected EOF in archive /usr/freeware/bin/tar: Error is not recoverable: exiting now # mt -f /hw/tape/tps21d1nrnsv rewind # mt -f /hw/tape/tps21d1nrnsv fsf 2 # dd if=/hw/tape/tps21d1nrnsv bs=32k skip=1 | /usr/freeware/bin/tar -tf - /usr/freeware/bin/tar: This does not look like a tar archive /usr/freeware/bin/tar: Skipping to next header /usr/freeware/bin/tar: Archive contains obsolescent base-64 headers 253158+0 records in 253158+0 records out /usr/freeware/bin/tar: Error exit delayed from previous errors Any ideas? jf
writing to tape
Good morning, I've got a 2.4.4 amanda on Solaris 9... We have this running well, consolidated all DLE to the single L9/LTO rather than splitting them across the L9/LTO (L9 - Sun StorEdge 9 slot jukebox) SDLT320, able to do this since we replaced amanda with a version with gtar compile option. We where a little backed up, the /maildb2/five DLE being held over and autoflushing the next day. Allowing only 3 tapes/run /maildb2/five was the last DLE dumped and was being left over for the following evening to be autoflushed... I allowed a 4th tape and reordered the dumporter, I wanted to start with the largest DLEs so they would get started, rather than starting close to start of business and running into the mid-afternoon. Unfortunately we seem to have started a dump to tape, rather than delaying for holding space wait. Is there a way to tell amanda that I'd prefer wait on holding area rather than allowing dump to tape ? Is there a method (perhaps in a newer version) that will allow some type of tapeorder, I could probably fill the tapes well with Ssss while I'd wanted to initiate the dumps as SSSsss. wcnotes:/ 0 4197472k finished (19:52:00) wcnotes:/export0 13843488k finished (2:54:50) wcnotes:/maildb1126322336k finished (7:21:00) wcnotes:/maildb2/five 1 66245170k dump done (5:14:29), wait for writing to tape wcnotes:/maildb2/four 1 11144610k writing to tape (7:21:11) wcnotes:/maildb2/one 1 50116350k dumping 1370368k ( 2.73%) (7:21:11) wcnotes:/maildb2/three 0 29249530k finished (1:12:39) wcnotes:/maildb2/two 1 41897850k dumping 1325152k ( 3.16%) (7:21:26) wcnotes:/usr1 0 293792k finished (19:52:39) thank you, Brian --- Brian R Cuttler [EMAIL PROTECTED] Computer Systems Support(v) 518 486-1697 Wadsworth Center(f) 518 473-6384 NYS Department of HealthHelp Desk 518 473-0773
Re: amtapetype - silly thing I tried
On Fri, Jun 01, 2007 at 08:00:21AM -0400, Jean-Louis Martineau wrote: Jon, amtapetype should terminate once it fill the disk. Jean-Louis I didn't see any smiley there ;) so I'll just note that once amtapetype seemed to recognize the file driver I half-expected it to also see the EOF like taper does. jl Jon LaBadie wrote: Back to back posts to the list on amtapetype and the file:driver made me wonder, will amtapetype work with virtual tapes? Not that it would be of any benefit, just wondered if it would. Well, to my surprise, amtapetype does seem to work with vtapes. But ... It recognized the slot and amanda tape in the slot, refusing to go on. It could be forced to overwrite the amanda tape in the slot with the -f option. It did its compression test. It wrote its files to the virtual tape slot. And wrote and wrote and wrote. I.e. unlike amdump/amflush that appear to see a pseudo EOF when their writing reached the 14GB size listed in the tapetype definition (and the estimate, -e option), amtapetype never sees the EOF. Of course it is trying to figure out what that size limit is supposed to be, so maybe it is reasonable that it never got an EOF ;) Not sure which surprised me most, that it started and worked well, or that it did not end normally. End of included message -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Amanda compression questions
My configuration:
zorn-[8] amadmin daily version
build: VERSION=Amanda-2.5.2-20070530
BUILT_DATE=Thu May 31 09:14:26 EDT 2007
BUILT_MACH=SunOS zorn.math.purdue.edu 5.10 Generic_118833-03 sun4u
sparc SUNW,Sun-Fire-280R
CC=/pkgs/gcc-3.4.3/bin/gcc
CONFIGURE_COMMAND='./configure' 'CC=/pkgs/gcc-3.4.3/bin/gcc'
'MT=/opt/csw/bin/mt' 'MTF=-f' 'MTX=/opt/csw/sbin/mtx'
'PERL=/opt/csw/bin/perl' '--prefix=/local/Amanda/amanda-2.5.2-20070530'
'--with-user=amanda' '--with-group=operator'
'--with-gnutar=/opt/csw/bin/gtar'
'--with-gnutar-listdir=/var/amanda/gnutar-lists'
'--with-includes=/opt/csw/include' '--with-libraries=/opt/csw/lib'
'--with-index-server=zorn' '--with-smbclient=/pkgs/samba/bin/smbclient'
'--with-tape-device=/dev/rmt/1bn' '--with-changer-device=/dev/changer/1'
'--with-maxtapeblocksize=2048'
My tape unit is a Sun C4 38 slot and 1 LTO2 tape drive. My tapetype is
defined as followed
define tapetype LTO2-HWC {
comment LTO-2-Hardware Compression on.
blocksize 1024 kbytes
length 30 mbytes #200G
filemark 0 kbytes
speed 27315 kps #27 Mb/s
}
length 30 mbytes was calculated by 1.5 times the actual size of 200GB.
Up to now my dumptype has had compress equal to none. I decided to let the
hardware do the compression work. The problem with this is the calculation
of the length is just a guess and seems to cause problems when trying to fit
files on the tape. In some cases amanda thinks it has more tape than it
really does so it tries to write a bigger file to the last part of the tape
and fails.
I am thinking of changing my setup to do software compression and see if
this will give me better tape usage.
Q1) I have asked Sun and the makers of the LTO2 drive and it seems there is
no way to turn hardware compression on or off on this unit, it is always on.
From my reading in the Amanda forum it seems that the LTO2 drive will sense
if the data is compressed and turn off hardware compression if the data is
already compressed. Is this correct?
Q2) Any suggestions, pros or cons in going this direction? I guess the trade
off is that I would get better tape usage at the expense of compute time to
do the software compression.
Q3) From the config.log it seems the default program for compression is gzip
and it uses -fast or -best. From gzip man page it says I can use -#x where x
is between 1 (fast) 9 (best). Is there a way to set the compress number
without having to use the compress server custom?
Thanks
Robert
_
Robert P. McGraw, Jr.
Manager, Computer System EMAIL: [EMAIL PROTECTED]
Purdue University ROOM: MATH-807
Department of MathematicsPHONE: (765) 494-6055
150 N. University Street FAX: (419) 821-0540
West Lafayette, IN 47907-2067
smime.p7s
Description: S/MIME cryptographic signature
Re: Amanda compression questions
McGraw, Robert P. wrote:
My configuration:
zorn-[8] amadmin daily version
build: VERSION=Amanda-2.5.2-20070530
BUILT_DATE=Thu May 31 09:14:26 EDT 2007
BUILT_MACH=SunOS zorn.math.purdue.edu 5.10 Generic_118833-03 sun4u
sparc SUNW,Sun-Fire-280R
CC=/pkgs/gcc-3.4.3/bin/gcc
CONFIGURE_COMMAND='./configure' 'CC=/pkgs/gcc-3.4.3/bin/gcc'
'MT=/opt/csw/bin/mt' 'MTF=-f' 'MTX=/opt/csw/sbin/mtx'
'PERL=/opt/csw/bin/perl'
'--prefix=/local/Amanda/amanda-2.5.2-20070530' '--with-user=amanda'
'--with-group=operator' '--with-gnutar=/opt/csw/bin/gtar'
'--with-gnutar-listdir=/var/amanda/gnutar-lists'
'--with-includes=/opt/csw/include' '--with-libraries=/opt/csw/lib'
'--with-index-server=zorn'
'--with-smbclient=/pkgs/samba/bin/smbclient'
'--with-tape-device=/dev/rmt/1bn'
'--with-changer-device=/dev/changer/1' '--with-maxtapeblocksize=2048'
My tape unit is a Sun C4 38 slot and 1 LTO2 tape drive. My tapetype is
defined as followed
define tapetype LTO2-HWC {
comment LTO-2-Hardware Compression on.
blocksize 1024 kbytes
length 30 mbytes #200G
filemark 0 kbytes
speed 27315 kps #27 Mb/s
}
“length 30 mbytes” was calculated by 1.5 times the actual size of
200GB.
Up to now my dumptype has had compress equal to none. I decided to let
the hardware do the compression work. The problem with this is the
calculation of the length is just a guess and seems to cause problems
when trying to fit files on the tape. In some cases amanda thinks it
has more tape than it really does so it tries to write a bigger file
to the last part of the tape and fails.
I am thinking of changing my setup to do software compression and see
if this will give me better tape usage.
Q1) I have asked Sun and the makers of the LTO2 drive and it seems
there is no way to turn hardware compression on or off on this unit,
it is always on. From my reading in the Amanda forum it seems that the
LTO2 drive will sense if the data is compressed and turn off hardware
compression if the data is already compressed. Is this correct?
Q2) Any suggestions, pros or cons in going this direction? I guess the
trade off is that I would get better tape usage at the expense of
compute time to do the software compression.
Q3) From the config.log it seems the default program for compression
is gzip and it uses –fast or –best. From gzip man page it says I can
use -#x where x is between 1 (fast) 9 (best). Is there a way to set
the compress number without having to use the “compress server custom”?
I'll let someone else comment on Q1 and Q3.
I've been using amanda 2.5.1p3 on a Sun E250 with Solaris 9 and a Sony
AIT5 16 slot changer. I set it up with server side software compression.
The AIT5 has a native capacity of 400G without compression, and that is
how I defined it in the tapetype. Since my E250 is dedicated to backup,
has dual processors, and has plenty of disk cache (two 300G lvd scsi
drives), I wasn't concerned about issues with having the server doing
compression. The result is almost perfect planning on the part of
amanda. After some experience with my DLEs, it has a very good idea how
things are going to turn out; and, after doing its own compression, it
knows exactly how much it is trying to put on a tape.
I think the trade-offs are way in favor of software compression. You
just never know how compressible something is going to be. Partitions
loaded with images that are in an already compressed format won't
compress. Web partitions with no images will compress beautifully.
Without some magical tape interface that gives you back how much
something compressed and how much tape is left, hardware compression
only leaves you the option of write until you hit eof, which makes it
impossible to decide whether you can fit a particular parition or not.
I'm presuming your 280R is dual processor and that you have given it
significant disk caching space.
---
Chris Hoogendyk
-
O__ Systems Administrator
c/ /'_ --- Biology Geology Departments
(*) \(*) -- 140 Morrill Science Center
~~ - University of Massachusetts, Amherst
[EMAIL PROTECTED]
---
Erdös 4
Re: amtapetype - silly thing I tried
Dustin J. Mitchell wrote: On Fri, Jun 01, 2007 at 10:40:38AM -0400, Jon LaBadie wrote: On Fri, Jun 01, 2007 at 08:00:21AM -0400, Jean-Louis Martineau wrote: Jon, amtapetype should terminate once it fill the disk. Jean-Louis I didn't see any smiley there ;) so I'll just note that once amtapetype seemed to recognize the file driver I half-expected it to also see the EOF like taper does. If I understand correctly (and just to summarize), Jon, you expected amtapetype to terminate when the tape was filled to its stated capacity (stated in the tapetype definition..) the way taper does. Instead, amtapetype runs until the OS returns ENOSPC (No space available) and then terminates, probably with a pretty accurate description of the amount of space available on the partition. Sound about right? Just an added comment: amtapetype is a diagnostic tool. It is hammering the device trying to determine what its limits are. It doesn't want a configuration file to tell it some limit that might not be right. It's going to figure it out for itself. The interesting thing here might be getting the performance of your drive, even though you are setting a lower limit for how much drive space you ultimately want amanda to use. --- Chris Hoogendyk - O__ Systems Administrator c/ /'_ --- Biology Geology Departments (*) \(*) -- 140 Morrill Science Center ~~ - University of Massachusetts, Amherst [EMAIL PROTECTED] --- Erdös 4
Re: amtapetype - silly thing I tried
On Fri, Jun 01, 2007 at 10:40:38AM -0400, Jon LaBadie wrote: On Fri, Jun 01, 2007 at 08:00:21AM -0400, Jean-Louis Martineau wrote: Jon, amtapetype should terminate once it fill the disk. Jean-Louis I didn't see any smiley there ;) so I'll just note that once amtapetype seemed to recognize the file driver I half-expected it to also see the EOF like taper does. If I understand correctly (and just to summarize), Jon, you expected amtapetype to terminate when the tape was filled to its stated capacity (stated in the tapetype definition..) the way taper does. Instead, amtapetype runs until the OS returns ENOSPC (No space available) and then terminates, probably with a pretty accurate description of the amount of space available on the partition. Sound about right? Dustin -- Dustin J. Mitchell Storage Software Engineer, Zmanda, Inc. http://www.zmanda.com/
Re: amtapetype - silly thing I tried
On Fri, Jun 01, 2007 at 12:49:32PM -0400, Chris Hoogendyk wrote: Dustin J. Mitchell wrote: On Fri, Jun 01, 2007 at 10:40:38AM -0400, Jon LaBadie wrote: On Fri, Jun 01, 2007 at 08:00:21AM -0400, Jean-Louis Martineau wrote: Jon, amtapetype should terminate once it fill the disk. Jean-Louis I didn't see any smiley there ;) so I'll just note that once amtapetype seemed to recognize the file driver I half-expected it to also see the EOF like taper does. If I understand correctly (and just to summarize), Jon, you expected amtapetype to terminate when the tape was filled to its stated capacity (stated in the tapetype definition..) the way taper does. Instead, amtapetype runs until the OS returns ENOSPC (No space available) and then terminates, probably with a pretty accurate description of the amount of space available on the partition. Sound about right? Dustin, Expected is not quite descriptive. I was first surprised to see that amtapetype even used the file driver semantics. Once it did, I was then curious, and yes, mildly surprised, that it did not honor the tapetype definition. Just an added comment: amtapetype is a diagnostic tool. It is hammering the device trying to determine what its limits are. It doesn't want a configuration file to tell it some limit that might not be right. It's going to figure it out for itself. The interesting thing here might be getting the performance of your drive, even though you are setting a lower limit for how much drive space you ultimately want amanda to use. Chris, While amtapetype was running those were exactly my thoughts. That it probably should go until the FS was full, but that if it did not, and stopped at the tapetype definition, then the performance info would indeed be useful. If one wanted a system-derived, rather than file: driver-drived capacity value, then one could simply set the tapetype definition to a monster number. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Amanda compression questions
Chris,
If I understand your question correctly...
Q1) I have asked Sun and the makers of the LTO2 drive and it seems
there is no way to turn hardware compression on or off on this unit,
it is always on. From my reading in the Amanda forum it seems that the
LTO2 drive will sense if the data is compressed and turn off hardware
compression if the data is already compressed. Is this correct?
The available tape devices are /dev/rmt/
unit{lmhu}[b][c][n]
Unit number 0, 1, 2...
Density - low, medium, high, ultra
b - byte swapping
c - compression
n - non-rewinding
I believe I've described it correctly, # man rmt will have more.
c is for HW compression, optional.
u - ultra is both high density and HW compression
With the tape type shown in the output below you should not be
compressing by default, but that may not be the device you are
actually writing to.
You are using tape device
'--with-tape-device=/dev/rmt/1bn'
but that is in the compilation, are you overwriting that in the
amanda.conf file with a specific tapedev parameter ?
I have also read on the amanda mailing list that if the data stream
is already compressed that the LTO series of drives will sense that
and disable, for the DLE HW compression. [I have not tested this nor
checked the docs for a source reference]
On Fri, Jun 01, 2007 at 12:36:15PM -0400, Chris Hoogendyk wrote:
McGraw, Robert P. wrote:
My configuration:
zorn-[8] amadmin daily version
build: VERSION=Amanda-2.5.2-20070530
BUILT_DATE=Thu May 31 09:14:26 EDT 2007
BUILT_MACH=SunOS zorn.math.purdue.edu 5.10 Generic_118833-03 sun4u
sparc SUNW,Sun-Fire-280R
CC=/pkgs/gcc-3.4.3/bin/gcc
CONFIGURE_COMMAND='./configure' 'CC=/pkgs/gcc-3.4.3/bin/gcc'
'MT=/opt/csw/bin/mt' 'MTF=-f' 'MTX=/opt/csw/sbin/mtx'
'PERL=/opt/csw/bin/perl'
'--prefix=/local/Amanda/amanda-2.5.2-20070530' '--with-user=amanda'
'--with-group=operator' '--with-gnutar=/opt/csw/bin/gtar'
'--with-gnutar-listdir=/var/amanda/gnutar-lists'
'--with-includes=/opt/csw/include' '--with-libraries=/opt/csw/lib'
'--with-index-server=zorn'
'--with-smbclient=/pkgs/samba/bin/smbclient'
'--with-tape-device=/dev/rmt/1bn'
'--with-changer-device=/dev/changer/1' '--with-maxtapeblocksize=2048'
My tape unit is a Sun C4 38 slot and 1 LTO2 tape drive. My tapetype is
defined as followed
define tapetype LTO2-HWC {
comment LTO-2-Hardware Compression on.
blocksize 1024 kbytes
length 30 mbytes #200G
filemark 0 kbytes
speed 27315 kps #27 Mb/s
}
?length 30 mbytes? was calculated by 1.5 times the actual size of
200GB.
Up to now my dumptype has had compress equal to none. I decided to let
the hardware do the compression work. The problem with this is the
calculation of the length is just a guess and seems to cause problems
when trying to fit files on the tape. In some cases amanda thinks it
has more tape than it really does so it tries to write a bigger file
to the last part of the tape and fails.
I am thinking of changing my setup to do software compression and see
if this will give me better tape usage.
Q1) I have asked Sun and the makers of the LTO2 drive and it seems
there is no way to turn hardware compression on or off on this unit,
it is always on. From my reading in the Amanda forum it seems that the
LTO2 drive will sense if the data is compressed and turn off hardware
compression if the data is already compressed. Is this correct?
Q2) Any suggestions, pros or cons in going this direction? I guess the
trade off is that I would get better tape usage at the expense of
compute time to do the software compression.
Q3) From the config.log it seems the default program for compression
is gzip and it uses ?fast or ?best. From gzip man page it says I can
use -#x where x is between 1 (fast) 9 (best). Is there a way to set
the compress number without having to use the ?compress server custom??
I'll let someone else comment on Q1 and Q3.
I've been using amanda 2.5.1p3 on a Sun E250 with Solaris 9 and a Sony
AIT5 16 slot changer. I set it up with server side software compression.
The AIT5 has a native capacity of 400G without compression, and that is
how I defined it in the tapetype. Since my E250 is dedicated to backup,
has dual processors, and has plenty of disk cache (two 300G lvd scsi
drives), I wasn't concerned about issues with having the server doing
compression. The result is almost perfect planning on the part of
amanda. After some experience with my DLEs, it has a very good idea how
things are going to turn out; and, after doing its own compression, it
knows exactly how much it is trying to put on a tape.
I think the trade-offs are way in favor of software compression. You
just never know how compressible something is going to be. Partitions
loaded with images that are in an already compressed format won't
compress. Web partitions with no images will compress
Amanda always flushes holding-disk
that. This
# is just a mechanism to stop Amanda trashing your network.
# Attributes are:
# use - bandwidth above which amanda won't start
# backups using this interface. Note that if
# a single backup will take more than that,
# amanda won't try to make it run slower!
define interface local {
comment a local disk
use 1000 kbps
}
define interface eth {
comment GBit ethernet
use 100 kbps
}
# You may include other amanda configuration files, so you can share
# dumptypes, tapetypes and interface definitions among several
# configurations.
#includefile /opt/amanda/etc/amanda/amanda.conf.main
amdump: start at Fr 1. Jun 19:48:57 CEST 2007
amdump: datestamp 20070601
amdump: starttime 20070601194857
planner: pid 12584 executable /opt/amanda/libexec/planner version 2.5.2
planner: build: VERSION=Amanda-2.5.2
planner:BUILT_DATE=Fr 1. Jun 16:32:17 CEST 2007
planner:BUILT_MACH=Linux kerberos 2.6.16 #1 SMP Thu Aug 3 12:46:53
CEST 2006 x86_64 GNU/Linux
planner:CC=gcc
planner:CONFIGURE_COMMAND='./configure' '--with-user=amanda'
'--with-group=amanda' '--without-bsd-security' '--without-ipv6'
'--with-krb5-security' '--with-buffered-dump' '--with-maxtapeblocksize=32'
'--with-gnuplot' '--prefix=/opt/amanda' '--with-config=Daily'
'--enable-threads=posix' '--with-gnutar=/opt/amanda/sbin/gtar-wrapper.pl'
planner: paths: bindir=/opt/amanda/bin sbindir=/opt/amanda/sbin
planner:libexecdir=/opt/amanda/libexec mandir=/opt/amanda/man
planner:AMANDA_TMPDIR=/tmp/amanda AMANDA_DBGDIR=/tmp/amanda
planner:CONFIG_DIR=/opt/amanda/etc/amanda DEV_PREFIX=/dev/
planner:RDEV_PREFIX=/dev/ DUMP=/sbin/dump
planner:RESTORE=/sbin/restore VDUMP=UNDEF VRESTORE=UNDEF
planner:XFSDUMP=/sbin/xfsdump XFSRESTORE=/sbin/xfsrestore
planner:VXDUMP=UNDEF VXRESTORE=UNDEF SAMBA_CLIENT=UNDEF
planner:GNUTAR=/opt/amanda/sbin/gtar-wrapper.pl
planner:COMPRESS_PATH=/bin/gzip UNCOMPRESS_PATH=/bin/gzip
planner:LPRCMD=/usr/bin/lpr MAILER=/usr/bin/Mail
planner:listed_incr_dir=/opt/amanda/var/amanda/gnutar-lists
planner: defs: DEFAULT_SERVER=kerberos DEFAULT_CONFIG=Daily
planner:DEFAULT_TAPE_SERVER=kerberos HAVE_MMAP NEED_STRSTR
planner:HAVE_SYSVSHM LOCKING=POSIX_FCNTL SETPGRP_VOID DEBUG_CODE
planner:AMANDA_DEBUG_DAYS=4 KRB5_SECURITY RSH_SECURITY
planner:CLIENT_LOGIN=amanda FORCE_USERID HAVE_GZIP
planner:COMPRESS_SUFFIX=.gz COMPRESS_FAST_OPT=--fast
planner:COMPRESS_BEST_OPT=--best UNCOMPRESS_OPT=-dc
READING CONF FILES...
driver: pid 12585 executable /opt/amanda/libexec/driver version 2.5.2
planner: timestamp 20070601194857
planner: time 0.002: startup took 0.002 secs
SENDING FLUSHES...
ENDFLUSH
SETTING UP FOR ESTIMATES...
planner: time 0.002: setting up estimates for kerberos:afs:user.testuser
driver: tape size 10485760
setup_estimate: kerberos:afs:user.testuser: command 0, options: none
last_level 1 next_level0 7 level_days 6getting estimates 0 (-2) 1 (-2) -1
(-2)
planner: time 0.002: setting up estimates took 0.000 secs
GETTING ESTIMATES...
driver: adding holding disk 0 dir /amanda/holding size 52428800 chunksize
2097152
reserving 0 out of 52428800 for degraded-mode dumps
driver: send-cmd time 0.088 to taper: START-TAPER 20070601194857
driver: started dumper0 pid 12588
driver: send-cmd time 0.088 to dumper0: START 20070601194857
driver: start time 0.089 inparallel 1 bandwidth 11000600 diskspace 52428800
dir OBSOLETE datestamp 20070601194857 driver: drain-ends tapeq FIRST
big-dumpers sssS
taper: pid 12587 executable taper version 2.5.2
taper: page size = 4096
taper: buffer size is 32768
dumper: pid 12588 executable dumper0 version 2.5.2
changer: opening pipe to: /opt/amanda/libexec/chg-disk -info
changer: opening pipe to: /opt/amanda/libexec/chg-disk -slot current
changer: opening pipe to: /opt/amanda/libexec/chg-disk -slot next
planner: time 0.236: got partial result for host kerberos disk
afs:user.testuser: 0 - -2K, 1 - -2K, -1 - -2K
taper: slot: 1 wrote label `TST001L0' date `20070601194857'
driver: result time 0.237 from taper: TAPER-OK
driver: state time 0.237 free kps: 11000600 space: 52428800 taper: idle
idle-dumpers: 1 qlen tapeq: 0 runq: 0 roomq: 0 wakeup: 0 driver-idle: not-idle
driver: interface-state time 0.237 if default: free 600 if local: free 1000
if eth: free 100
driver: hdisk-state time 0.237 hdisk 0: free 52428800 dumpers 0
planner: time 0.298: got partial result for host kerberos disk
afs:user.testuser: 0 - 1616K, 1 - -2K, -1 - -2K
planner: time 0.390: got partial result for host kerberos disk
afs:user.testuser: 0 - 1616K, 1 - 150K, -1 - -2K
planner: time 0.391: got result for host kerberos disk afs:user.testuser: 0 -
1616K, 1 - 150K, -1 - -2K
planner: time 0.392: getting estimates took 0.389 secs
FAILED QUEUE: empty
DONE QUEUE:
0: kerberos
Re: Amanda always flushes holding-disk
I'm always happy to see that amanda do what it should do. amanda always write up to runtapes tapes if it has data to write. Why do you configure a tapedev if you don't what to use it? Try the following command if you don't want to use a tape: amdump conf -o tapdev=/no/such/device -o tpchanger= Jean-Louis Harald Schioeberg wrote: Hi, i have the problem, that amanda always flushes its holding-disk, no matter what i do. I tried 2.5.2 and 2.5.2-flush-1. I tried with different reserves and holdingdisk set to auto and required
Re: autoflush doesn't?
On Thursday 31 May 2007, Dustin J. Mitchell wrote:
if (strcmp(dl-name, workdir-d_name) == 0) {
I note this patch didn't make it into the Amanda 2.5.2-20070531 snapshot.
So I did it before I built it. Install amcheck was normal.
--
Cheers, Gene
There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order.
-Ed Howdershelt (Author)
If only Dionysus were alive! Where would he eat?
-- Woody Allen
Re: Amanda always flushes holding-disk
Jean-Louis Martineau wrote: I'm always happy to see that amanda do what it should do. Ok, but what's the autoflush off parameter good for? harald
Re: Amanda always flushes holding-disk
On Fri, Jun 01, 2007 at 09:10:16PM +0200, Harald Schioeberg wrote: Jean-Louis Martineau wrote: I'm always happy to see that amanda do what it should do. Ok, but what's the autoflush off parameter good for? Flushing deals with dumps that were left in the holding disk on a previous run of amdump. Possibly because of a hardware difficulty or because of insufficient tape capacity. Or because some people just leave the tape out. By default these must be flushed manually with the amflush command. Autoflush yes allows amanda to automatically flush them as part of the NEXT amdump run. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: autoflush doesn't?
On Fri, Jun 01, 2007 at 02:31:27PM -0400, Gene Heskett wrote:
On Thursday 31 May 2007, Dustin J. Mitchell wrote:
if (strcmp(dl-name, workdir-d_name) == 0) {
I note this patch didn't make it into the Amanda 2.5.2-20070531 snapshot.
The general practice has been to post fixes and wait for confirmation
that they corrected the problem before committing them to subversion.
Sorry if that caused some confusion here.
So I did it before I built it. Install amcheck was normal.
Great! Let me know how the amflush and/or autoflush go.
Dustin
--
Dustin J. Mitchell
Storage Software Engineer, Zmanda, Inc.
http://www.zmanda.com/
amanda-20070530 amrecover problem with restoring from older amanda version.
My configuration: build: VERSION=Amanda-2.5.2-20070530 BUILT_DATE=Thu May 31 09:14:26 EDT 2007 BUILT_MACH=SunOS zorn.math.purdue.edu 5.10 Generic_118833-03 sun4u sparc SUNW,Sun-Fire-280R CC=/pkgs/gcc-3.4.3/bin/gcc CONFIGURE_COMMAND='./configure' 'CC=/pkgs/gcc-3.4.3/bin/gcc' 'MT=/opt/csw/bin/mt' 'MTF=-f' 'MTX=/opt/csw/sbin/mtx' 'PERL=/opt/csw/bin/perl' '--prefix=/local/Amanda/amanda-2.5.2-20070530' '--with-user=amanda' '--with-group=operator' '--with-gnutar=/opt/csw/bin/gtar' '--with-gnutar-listdir=/var/amanda/gnutar-lists' '--with-includes=/opt/csw/include' '--with-libraries=/opt/csw/lib' '--with-index-server=zorn' '--with-smbclient=/pkgs/samba/bin/smbclient' '--with-tape-device=/dev/rmt/1bn' '--with-changer-device=/dev/changer/1' '--with-maxtapeblocksize=2048' paths: bindir=/local/Amanda/amanda-2.5.2-20070530/bin sbindir=/local/Amanda/amanda-2.5.2-20070530/sbin libexecdir=/local/Amanda/amanda-2.5.2-20070530/libexec mandir=/local/Amanda/amanda-2.5.2-20070530/man AMANDA_TMPDIR=/tmp/amanda AMANDA_DBGDIR=/tmp/amanda CONFIG_DIR=/local/Amanda/amanda-2.5.2-20070530/etc/amanda DEV_PREFIX=/dev/dsk/ RDEV_PREFIX=/dev/rdsk/ DUMP=/usr/sbin/ufsdump RESTORE=/usr/sbin/ufsrestore VDUMP=UNDEF VRESTORE=UNDEF XFSDUMP=UNDEF XFSRESTORE=UNDEF VXDUMP=UNDEF VXRESTORE=UNDEF SAMBA_CLIENT=/pkgs/samba/bin/smbclient GNUTAR=/opt/csw/bin/gtar COMPRESS_PATH=/bin/gzip UNCOMPRESS_PATH=/bin/gzip LPRCMD=/local/bin/lpr MAILER=/usr/ucb/Mail listed_incr_dir=/var/amanda/gnutar-lists defs: DEFAULT_SERVER=zorn DEFAULT_CONFIG=DailySet1 DEFAULT_TAPE_SERVER=zorn DEFAULT_TAPE_DEVICE=/dev/rmt/1bn NEED_STRSTR HAVE_SYSVSHM LOCKING=**NONE** SETPGRP_VOID DEBUG_CODE AMANDA_DEBUG_DAYS=4 BSD_SECURITY RSH_SECURITY USE_AMANDAHOSTS CLIENT_LOGIN=amanda FORCE_USERID HAVE_GZIP COMPRESS_SUFFIX=.gz COMPRESS_FAST_OPT=--fast COMPRESS_BEST_OPT=--best UNCOMPRESS_OPT=-dc I tested amrecover on a file that was backed up with amanda-20070530 and had no problem. Today I tried to do a amrecover on a file the was backed up with amanda-2.5.1p2 and get the following: Extracting files using tape drive /dev/rmt/1bn on host zorn. Load tape D5 now Continue [?/Y/n/s/t]? y Not an amanda tape Looking for tape D5... Not an amanda tape Load tape D5 now Continue [?/Y/n/t]? y Not an amanda tape Not an amanda tape Looking for tape D5... ^[[ANot an amanda tape Load tape D5 now Continue [?/Y/n/t]? y Continue [?/Y/n/t]? Not an amanda tape Not an amanda tape Looking for tape D5... Not an amanda tape Load tape D5 now Continue [?/Y/n/t]? n The following is from amrecover.20070601151822.debug: amidxtaped_streams[0].fd = 810a0 amrecover: time 353.630: security_streaminit(stream=8bf88, driver=ff2d8f6c (BSD)) amrecover: time 353.635: connect_port: Try port 0: Available - amrecover: time 353.635: connected to 128.210.3.177.50856 amrecover: time 353.635: our side is 0.0.0.0.50860 amrecover: time 353.635: try_socksize: send buffer size is 65536 amrecover: time 353.635: try_socksize: receive buffer size is 65536 amidxtaped_streams[1].fd = 8bf88 amrecover: time 353.635: security_close(handle=7d2f8, driver=ff2d8f6c (BSD)) amrecover: time 581.477: security_stream_close(8bf88) amrecover: time 581.478: Can't read file header amrecover: time 581.479: pid 2521 finish time Fri Jun 1 15:28:03 2007 amrecover: time 581.484: security_stream_close(810a0) amrecover: time 822.433: security_stream_close(732d8) amrecover: time 822.434: pid 1863 finish time Fri Jun 1 15:32:04 2007 which seems to fit Not an amanda tape message from the amrecover script. I do not remember seeing anything about this any of the amanda-users forum but could have over looked. Robert _ Robert P. McGraw, Jr. Manager, Computer System EMAIL: [EMAIL PROTECTED] Purdue University ROOM: MATH-807 Department of MathematicsPHONE: (765) 494-6055 150 N. University Street FAX: (419) 821-0540 West Lafayette, IN 47907-2067 smime.p7s Description: S/MIME cryptographic signature
Re: amanda-20070530 amrecover problem with restoring from older amanda version.
What's in the amidxtaped.*.debug on the server? Jean-Louis McGraw, Robert P. wrote: My configuration: build: VERSION=Amanda-2.5.2-20070530 BUILT_DATE=Thu May 31 09:14:26 EDT 2007 BUILT_MACH=SunOS zorn.math.purdue.edu 5.10 Generic_118833-03 sun4u sparc SUNW,Sun-Fire-280R CC=/pkgs/gcc-3.4.3/bin/gcc CONFIGURE_COMMAND='./configure' 'CC=/pkgs/gcc-3.4.3/bin/gcc' 'MT=/opt/csw/bin/mt' 'MTF=-f' 'MTX=/opt/csw/sbin/mtx' 'PERL=/opt/csw/bin/perl' '--prefix=/local/Amanda/amanda-2.5.2-20070530' '--with-user=amanda' '--with-group=operator' '--with-gnutar=/opt/csw/bin/gtar' '--with-gnutar-listdir=/var/amanda/gnutar-lists' '--with-includes=/opt/csw/include' '--with-libraries=/opt/csw/lib' '--with-index-server=zorn' '--with-smbclient=/pkgs/samba/bin/smbclient' '--with-tape-device=/dev/rmt/1bn' '--with-changer-device=/dev/changer/1' '--with-maxtapeblocksize=2048' paths: bindir=/local/Amanda/amanda-2.5.2-20070530/bin sbindir=/local/Amanda/amanda-2.5.2-20070530/sbin libexecdir=/local/Amanda/amanda-2.5.2-20070530/libexec mandir=/local/Amanda/amanda-2.5.2-20070530/man AMANDA_TMPDIR=/tmp/amanda AMANDA_DBGDIR=/tmp/amanda CONFIG_DIR=/local/Amanda/amanda-2.5.2-20070530/etc/amanda DEV_PREFIX=/dev/dsk/ RDEV_PREFIX=/dev/rdsk/ DUMP=/usr/sbin/ufsdump RESTORE=/usr/sbin/ufsrestore VDUMP=UNDEF VRESTORE=UNDEF XFSDUMP=UNDEF XFSRESTORE=UNDEF VXDUMP=UNDEF VXRESTORE=UNDEF SAMBA_CLIENT=/pkgs/samba/bin/smbclient GNUTAR=/opt/csw/bin/gtar COMPRESS_PATH=/bin/gzip UNCOMPRESS_PATH=/bin/gzip LPRCMD=/local/bin/lpr MAILER=/usr/ucb/Mail listed_incr_dir=/var/amanda/gnutar-lists defs: DEFAULT_SERVER=zorn DEFAULT_CONFIG=DailySet1 DEFAULT_TAPE_SERVER=zorn DEFAULT_TAPE_DEVICE=/dev/rmt/1bn NEED_STRSTR HAVE_SYSVSHM LOCKING=**NONE** SETPGRP_VOID DEBUG_CODE AMANDA_DEBUG_DAYS=4 BSD_SECURITY RSH_SECURITY USE_AMANDAHOSTS CLIENT_LOGIN=amanda FORCE_USERID HAVE_GZIP COMPRESS_SUFFIX=.gz COMPRESS_FAST_OPT=--fast COMPRESS_BEST_OPT=--best UNCOMPRESS_OPT=-dc I tested amrecover on a file that was backed up with amanda-20070530 and had no problem. Today I tried to do a amrecover on a file the was backed up with amanda-2.5.1p2 and get the following: Extracting files using tape drive /dev/rmt/1bn on host zorn. Load tape D5 now Continue [?/Y/n/s/t]? y Not an amanda tape Looking for tape D5... Not an amanda tape Load tape D5 now Continue [?/Y/n/t]? y Not an amanda tape Not an amanda tape Looking for tape D5... ^[[ANot an amanda tape Load tape D5 now Continue [?/Y/n/t]? y Continue [?/Y/n/t]? Not an amanda tape Not an amanda tape Looking for tape D5... Not an amanda tape Load tape D5 now Continue [?/Y/n/t]? n The following is from amrecover.20070601151822.debug: amidxtaped_streams[0].fd = 810a0 amrecover: time 353.630: security_streaminit(stream=8bf88, driver=ff2d8f6c (BSD)) amrecover: time 353.635: connect_port: Try port 0: Available - amrecover: time 353.635: connected to 128.210.3.177.50856 amrecover: time 353.635: our side is 0.0.0.0.50860 amrecover: time 353.635: try_socksize: send buffer size is 65536 amrecover: time 353.635: try_socksize: receive buffer size is 65536 amidxtaped_streams[1].fd = 8bf88 amrecover: time 353.635: security_close(handle=7d2f8, driver=ff2d8f6c (BSD)) amrecover: time 581.477: security_stream_close(8bf88) amrecover: time 581.478: Can't read file header amrecover: time 581.479: pid 2521 finish time Fri Jun 1 15:28:03 2007 amrecover: time 581.484: security_stream_close(810a0) amrecover: time 822.433: security_stream_close(732d8) amrecover: time 822.434: pid 1863 finish time Fri Jun 1 15:32:04 2007 which seems to fit “Not an amanda tape” message from the amrecover script. I do not remember seeing anything about this any of the amanda-users forum but could have over looked. Robert _ Robert P. McGraw, Jr. Manager, Computer System EMAIL: [EMAIL PROTECTED] Purdue University ROOM: MATH-807 Department of Mathematics PHONE: (765) 494-6055 150 N. University Street FAX: (419) 821-0540 West Lafayette, IN 47907-2067
RE: Amanda always flushes holding-disk
Hi, Harald, -- What Jon said is correct, but I will rephrase from a different viewpoint Amanda goes like this: - grab data from where it lives - copy to holding disk - move on to tape: - if there is a tape, and there's room on it, and so on - move in large chunks, possibly tape sized, so that the tape never stops So if there is a problem with the tape, you still do the backup, but it only gets as far as the holding disk. One advantage of this, if you have a large holding disk, is that it is moved across the network at the expected time when network traffic is probably lower. Then amflush is used to move the data from the holding disk to the tape when you resolve whatever problem stopped it going to tape in the first place. -- Next point: Why did you expect the other behavior? Are you trying to do disk-to-disk-to-tape backups? Or just using Amanda to backup to a disk? These can both be done, but it's slightly more configuration. I am doing disk-to-disk backup to 250GB disks, then moving the disks (in USB external cases) to off-site storage. Look ma, no tapes! For backups to disk, see: http://www.amanda.org/docs/howto-filedriver.html http://wiki.zmanda.com/index.php/File_driver Rob -- Rob Echlin Software Development Environment Prime Espial IPTV rechlin -at- espial.com Phone: +1 613-230-4770 ext 1150 www.espial.com Espial Group Inc. Confidential Important Notice: This communication is intended to be received only by the individual or entity to whom or to which it is addressed and may contain information that is privileged, confidential and/or subject to copyright. Any unauthorized use, copying, review or disclosure of this communication is strictly prohibited. If you have received this communication in error, please delete the message and notify the sender by reply email. Thank you for your cooperation. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Harald Schioeberg Sent: Friday, June 01, 2007 3:10 PM To: Jean-Louis Martineau Cc: amanda-users@amanda.org Subject: Re: Amanda always flushes holding-disk Jean-Louis Martineau wrote: I'm always happy to see that amanda do what it should do. Ok, but what's the autoflush off parameter good for? harald
