Re: impressive :)
On 2011-02-26 03:17, Charles Curley wrote: What would *really* save space is for Amanda to detect that machines A, B and C all have the save version of file foo (even if they are in different places), and only actually keep one around. I suspect that would be very compute intensive at backup time, but these days That is what BackupPC does... At the expense of having the backup in a filesystem currently... And even that can be eliminated by having only meta data and checksums (sha-md5.. whatever) online on disk. (and then again at the expense of a much more complicated backend, where it would be much more complicated to restore backups when your backup server has crashed etc.) -- Paul Bijnens, XplanationTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Backup the same data to two vTapes
On 2010-12-30 18:52, Marc Muehlfeld wrote: currently I backup to a storage system and copy it to a second one to ensure, that if it breaks, I still have my vTapes on the second one. Is it possible to configure Amanda to write backuped data to two vTape-Libraries at the same time (the same data to two vTapes)? Here is what I wrote about it a few years ago: http://wiki.zmanda.com/index.php/How_To:Mirror_Dumps_to_Virtual_Tapes_and_Real_Tapes_using_RAIT However, the page could use some updates for the current version of Amanda. -- Paul Bijnens, XplanationTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Backup the same data to two vTapes
On 2010-12-31 10:35, Marc Muehlfeld wrote: Hello, Am 31.12.2010 10:19, schrieb Paul Bijnens: Here is what I wrote about it a few years ago: http://wiki.zmanda.com/index.php/How_To:Mirror_Dumps_to_Virtual_Tapes_and_Real_Tapes_using_RAIT This seems to be a good solution. I'll try it. But one question about it: What happens, if one of my storage is down during backup? Is then the data written to the other one and kept in the holding disk for flushing to later to the first one? How amanda handles this situation? This explains some more about the RAIT, and the differences between RAID and RAIT: http://wiki.zmanda.com/man/amanda-devices.7.html (Section RAIT Device) and indeed, the handling of error-modes is different. Unlike RAID, a RAIT will not automatically handle the failure of one of components. A RAID system will continue functioning in a degraded state, while the RAIT device will not write in a degraded state. It does allow reading in a degraded state, in order to support recoveries. But you can of course change the tapedev string while the disksystem is being repaired, and continue using Amanda. -- Paul Bijnens, XplanationTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Nitpicks? -- permissions
On 2010-09-12 17:25, Dustin J. Mitchell wrote: On Sat, Sep 11, 2010 at 10:42 PM, Lisa Seelye l...@thedoh.com wrote: Disk permissions and file permissions are always a nightmare. Some things aren't right and amcheck complains about each one in turn. It would be useful to have some kind of thorough lint mode, perhaps with a --fix-permissions to repair the permissions (when run with root). Ideally amcheck would be finding all of the permissions problems on the first run - although sometimes one permission problem can make it impossible to find the next. Is there some specific condition that you're seeing appear only on the second run? As for adding a --fix-permissions option is an interesting idea, but I worry that it would blindly 'mkdir' and 'chown' and 'chmod' things that you might not want it to. Note that many missing directories are created at first run. If some of those are missing, we could fix that up. Do you have a particular example you'd like me to look at? I remember a few requests for help on this list, when someone ran the very first time as user root, so all the folders and files were created as root. Maybe just to do some test run the very first time, while trying to get it compiled etc. When then later trying to run as the amandabackup user you're in trouble, as e.g. even the debug folder is not even writable by amanda and you have no immediate clue about what is going wrong. -- Paul Bijnens, XplanationTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: lvm snapshots
Hi Stefan and fellow amanda-users, Catching up on my mail... Just to confirm that you may post the script. Anybody even use it and adapt it to his needs. It's free software (as in free beer and in free speech). Although the script is a bit dated for the current amanda... (Still need to catch up on that too.) On 2010-08-27 21:14, Stefan G. Weichinger wrote: Am 27.08.2010 20:11, schrieb Jon LaBadie: I am thinking of backing up some DLEs that are fairly active from a snapshot rather than from the filesystem directly. I looked for documentation on doing this but could find none, nor any HowTo's. Anyone know of any docs on backing up from snapshots? (and the receprocal restore :) I use a script written by our fellow amanda-user Paul Bijnens (thanks again!). I only slightly modified it regarding XFS. This script is from times when amanda had no script-API, so it would maybe be time to somehow rewrite it ... Right now I run it in advance of the actual amdump via crontab. I assume Paul would allow me to share the script (maybe it has been posted on-list already?) ... Paul? ;-) Stefan -- Paul Bijnens, XplanationTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Backup Barracuda Message Archiver 450 via CIFS/Samba
For doing the backup with samba, three machine are involved: - the amanda server, who contacts the amanda client - the amanda client, who runs smbclient, which contacts the SMB share - the machine that needs the backup, running CIFS/samba/native-Windows-share-stuff Many people configure their setup for this with client and server the same machine; but it must not be like that, e.g. to offload the load client software compression to another host. In the disklist you have: name-of-amanda-client //pcname/sharename dymptype-with-samba Note that the backups you are making are actually coming from the host pcname; the amanda-client is just a passthrough. Nothing prevents you using the hostname of the server itself in the name-of-amanda-client. Is that what you were looking for? On 2010-02-01 16:51, Corey Jones wrote: Hello All, I'm trying to backup two Barracuda Message Archiver 450 via CIFS/Samba and I was wondering if anyone have or know of any resource/examples of this type of setup. The main problem is (I think) the /etc/amandapass file is read from the amanda client only. I need to add the username:password/WORKGROUP information to the server side of amanda; to connect to the Barracuda CIFS/SAMBA service running on the appliance So, basically... I want the Amanda server to connect to the CIFS/Samba server running on the Barracuda and have the Amanda server pass the username:password/WORKGROUP to the Barracuda and successfully backup the message archive. Thanks in advance, Corey -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: [Amanda-users] Newbie - Confused on How to Backup My WinXP PC to USB Disk
On 11/18/2009 12:02 AM, Stefan G. Weichinger wrote: Paul Bijnens schrieb: On 2009-11-17 02:18, Yadda wrote: Total Newbie with Amanda. I only want to backup my WinXP PC to an external USB drive and am lost on how to set this up. Please help!!! :( Airports are where airplaines land, not a place to park your bicycle. [...] (Ok, I'm parking my wifes bike on that runway too, but just because that runway is already there, not because it's the best bike park). respect, Paul, what an analogy ;-) http://www.bing.com/maps/?v=2cp=sggbbshc5fs4scene=21014403lvl=2sty=b Right in the center you see the bike park, for about 15 bicycles even, with a transparant roof. And when you move the scene a little bit to the northwest, you can see the DHL plane loaded with Amanda tapes for offshore storage.
Re: [Amanda-users] Newbie - Confused on How to Backup My WinXP PC to USB Disk
On 2009-11-17 02:18, Yadda wrote: Total Newbie with Amanda. I only want to backup my WinXP PC to an external USB drive and am lost on how to set this up. Please help!!! :( Airports are where airplaines land, not a place to park your bicycle. You're looking in the wrong software. Amanda is aimed for a network of computers which are trying to do their backups over the network on a central server, which has lots of capacity in disk and/or tape. And that server is best some modern Unix variant as well :-). (Ok, I'm parking my wifes bike on that runway too, but just because that runway is already there, not because it's the best bike park). -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Another oddity from Gene
On 2009-10-19 16:34, Gene Heskett wrote: On Monday 19 October 2009, Dustin J. Mitchell wrote: On Mon, Oct 19, 2009 at 8:05 AM, Gene Heskett gene.hesk...@verizon.net wrote: if [`/bin/ls /dumps`]; then line 173 This can be a bit tricky in portable shell, but maybe this is closer to what you want: if [ `/bin/ls /dumps` != '' ] Dustin I'll try that, but IIRC if the dir is empty, ls still returns a linefeed. brb. Nope, tried with !='' and !='\n', with almost the same result, the diff being that now the error message on the console is minus the ending ] now. ./flush.sh: line 173: [20091019011512: command not found In portable shell, there must be space before and after the '[' and ']' as well as before and after the comparison operator. I fail to see why bash is looking for that ls return as a command to be executed, almost like bash is busted, and its been ages since bash was updated. This script has worked many times since the last time I saw a bash update go by. its enough to make a fellow wish it was beer-thirty and I haven't even finished my first cup of tea! -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: mailing list issues (and OpenBSD clients)
On 2009-08-27 15:07, stan wrote: I seem to be unable to get a failry lengthy message I wrote about the issues I am having with OpenBSD clinets acepted by the listserver. I've tried submiting it with and withou tacahments, and resubscribing to the list with no luck. But a shor message to the listserver cam abck. Strange. In any acse, in addition to the atachments, i ahve now put the email up on http://becahcave.net/amanda Hopefully thos interested in the issue with OpenBSD clinets will atke a minute to read it. Thanks. offtopic :-) Are you 100% sure you didn't -- by accident -- made a typo in your name when subscribing to the list :-) /offtopic -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: OpenBSD issues
On 2009-08-27 15:16, stan wrote: On Thu, Aug 27, 2009 at 08:49:21AM -0400, stan wrote: Sorry again. I just resubscribed, and I am trying to make certain I am getting mail from the list. This is truning into a _verY- frustrating morning! I have made several attempts to send to the list some information regarding the status of OpenBSD clients, and NONE of the attempts have come back to me. I have posted 2 short test messages (this is a reply to the 2nd one) and they have BOTH shown up. I have resubscribed to the lsit, but that did not help. I have put the original message, and the debug files that I tried to attach to it up at http://bechacave.net/amanda not there, and neither at On 2009-08-27 15:07, stan wrote: Strange. In any acse, in addition to the atachments, i ahve now put the email up on http://becahcave.net/amanda but here: http://beachcave.net/amanda (to solve others from trying some more anagrams) I would greatly appreciate anyone that might have any ideas as to how to go foward on debuging this taking a minute, and looking at this. Thanks! -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: filesys for tape?
On 07/21/2009 06:11 PM, Dustin J. Mitchell wrote: On Tue, Jul 21, 2009 at 11:52 AM, rirvze34...@verizon.net wrote: What filesystem and mount options are recommended for a tapetype harddisk? Details: The amanda-server is a Debian 5 (lenny) host. The tape size is 40 gigs. The filesystems are about 540, 990, and 950 gigs. Not that it much matters but I run most everything on ext3 filesystems with the relatime option (or noatime if older). I've never tested any numbers on which filesystem best suits a VFS device. Personally, I use ext3, but I would certainly be interested in some empirical analysis of this topic. Removing a large file on ext3 takes ages very long, and the backup images on the harddisk are all very large usually. The noatime or relatime option will not matter much here. The new ext4, when using extends, removes large files much faster. I have personnaly no experience with XFS, but that is also supposedly be better in handling very large files. Otherwise ext3 is good enough. You want stability first; removing the files is still faster than rewinding a physical tape. -- Paul
Re: Amanda, separable but related problems, cross architecture.
On 07/03/2009 05:49 PM, Brian Cuttler wrote:
I am perfroming dump from Solaris 10 x86, using amanda 2.6.1
to an SL24/LTO4.
The following is cronological, some of the issues are separable.
The particular client with the issue is a MAC server with over
250Gig of data.
The dumps where taking a _lot_ time, so I tried to divide the DLE
into 2 separate DLEs. Of of those was still large, so I tried to
divide it into 2 again.
Rather than dumping /, I was dumping /Users and /trel, trel was _large_
over 250 gig by itself.
trel:/trel cssadmin$ ls
Active Staff FoldersFunded Programs Quality Assurance
ApplicationsGraphpadReferences
Archived StudiesImages Research Studies
ArchivesLINCTemporary Items
DRC+ PE07 Blood Lab Operations Trace Elements
DRCII PE10 UrineLead Poisoning Trash
DRCII PE12 Research NYS PT Program
External PT
I attempted the following in the disklist
trel /Users comp-user-tar
#trel /treluser-tar
trel /trelAM /trel {
comp-user-tar
exclude [N-Z]*
}
trel /trelNZ /trel {
comp-user-tar
exclude [A-M]*
}
Which naturally, since there where 'new' DLEs attempted level 0 dumps
on both.
However the combined for the two new DLEs was not the expected 250G but
only about 50 Gig.
Those exclude statements are not doing what you expect.
There is an assymetry in Amanda's use of include and
exclude, due to the how gtar implements both features
For gnutar, the arguments for exclude are glob patterns
that are applied to each file *part* (basename).
Assuming a file tree like:
./
./References/
./References/Zdir/
./References/Adir/
./References/Adir/Zfile
./Archives/
./Archives/Ydir/
./Archives/Bdir/
then using gnutar like this shows the file which will be
included:
$ tar -cvf /dev/null --exclude A*
./
./References/
./References/Zdir/
You noticed? Not only Archived, but also the subfolder Adir matches
the exclude pattern (on the basename of the pathname) and is omitted.
But anchoring the pattern to the start of the path works, just like
in the shell:
$ tar -cvf /dev/null --exclude=./A* .
./
./References/
./References/Zdir/
./References/Adir/
./References/Adir/Zfile
So in your exclude scheme you exclude once ALL the files
and folders start with an uppercase A-M in one case and N-Z in
the other case. That means that you will miss files that happen to
a subfolder of the other class each time!
Consider a folder like References/Adir: the contents will be
omitted each time. Because when excluding [A-M] the whole folder
Adir will be omitted and when excluding [N-Z] will exclude
References as a whole.
The implementation of include in gnutar is completely different.
First gnutar does NOT consider the arguments as globs. Actually
the include is just like naming multiple toplevel folders (like the .
in my example above).
Having a glob on the other hand is a nice feature, so Amanda actually
expands the arguments as glob (limiting to only 1 level!) and hands
the expanded list to gnutar.
That is why this would work as you intended (changing some other
details that I find easier):
trel /trel/./_AM_ /trel {
comp-user-tar
include ./[A-M]*
}
trel /trel/./_NZ_ /trel {
comp-user-tar
include ./[N-Z]*
}
trel /trel/./_REST_ /trel {
comp-user-tar
exclude append ./[A-M]*
exclude append ./[N-Z]*
}
- I use the selfdocumenting notation /top/level/folder/./_mnemonic_ with
the whole toplevel folder first, followed by a separator /./, followed by a
mnemonic that summarizes the contents.
- The exclude patterns *always* starts with ./ to anchor them
to the toplevel, and get the symmetry with the include syntax, as implemented
by Amanda back.
- And I *always* add a _REST_ that carefully uses exclude append with exactly
the same exclude patterns (that have then indeed the same meaning).
You never know how inventive people (or yourself) can be when creating
unforseen toplevel folders, like lowercase, numbers, high-ascii codes, spaces,
control chars etc.
--
Paul
Re: Amanda's planning for use of holding disk space
On 2009-06-30 19:52, Chris Hoogendyk wrote: So, the question is, if Amanda has more than one holding disk (partition), and they differ in size, will Amanda know when the smaller one is inadequate for a particular DLE and explicitly choose the larger one? Also, if I have specified spindle numbers in my disklist, so that Amanda will avoid doing parallel dumps from the same spindle, is there any way of informing Amanda of the spindle numbers for the holding disks (partitions) and taking that into account in the planning? No problem at all. You make config entry for each holdingdisk. On each holdingdisk you can specify peculiarities of the disk, like the path to the toplevel holdingdirectory, and the amount of free space that should not be used etc. Make sure to specify a chunksize that will fit easily on the smallest disk. I make my chunksize 1GB. Amanda will spread the holdingdisk data over all the holdingdisks avoiding any problems when one DLE would not fit on a single area. As side benefit you'll also get some improvement in throughput because now taper reading finished dump images will compete less with the dumpers writing to disk. Making it different filesystems instead of one large logical volume makes future adding/removing/swapping disks easier as well. And having a disk error on one of the disks in raid0 lvm's is much worse than on independent filesystems. -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: exclude lists
On 2009-05-19 15:35, Brandon Metcalf wrote: m == martin...@zmanda.com writes: m m m Seems like a nice feature would be to allow the DLE to be specified in m the exclude list. m m The 'exclude list' can be a relative path to the dle. So, you're saying I can specify one dumptype with exclude list ./amanda/DailySet1/exclude-list or is it exclude list amanda/DailySet1/exclude-list and have this file exist on each DLE? Specify it as: exclude list optional append .exclude-list And have a file (optionally) in the directory of each DLE, e.g. /.exclude-list (only for the root DLE) /var/.exclude-list (only for the /var DLE) /space/.exclude-list (only for the /space DLE) Have a look in the man page for the exact meaning of optional and append. -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: restore over multiple tape
On 2009-05-19 16:52, Franck GANACHAUD wrote: I didn't specify these. Original diskset is 250Gb to save to DLT4 35GB tapes using a 8 slots autoloader. I'm going to split this diskset going down one level in the filesystem which will give me a little less than 150 diskset. I just have to think about a mechanism to upgrade the diskset list automaticaly. Try to do it like this: http://wiki.zmanda.com/index.php/How_To:Split_DLEs_With_Exclude_Lists And your last problem is solved (and probably your first as well reducing 150 disksets to a little less). -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: support for tape library sun sl24 or sun sl48
On 2009-05-18 16:28, Uwe Bartels wrote:
Hi,
does anybody know if the tape library sun(storagetek) sl24 or
sun(storagetek) sl48 with 2 lto4 tape drives are working?
If so. how are the general parameters, which i'm missing in the Tape
Type List http://wiki.zmanda.com/index.php/Tapetype%20definitions .
http://wiki.zmanda.com/index.php/Tapetype_definitions#LTO4
Dell PowerVault LTO4-120 - LTO4-800 tape
define tapetype LTO4 {
comment Dell LTO4 800Gb - Compression Off
length 802816 mbytes
filemark 0 kbytes
speed 52616 kps
}
Arcvault 48 - HP LTO4-800 tape
define tapetype HPLTO4 {
comment HP LTO4 800gb - Compression Off
length 772096 mbytes
filemark 0 kbytes
speed 31368 kps
}
But I believe both of these entries have a speed that is indicating
the host cannot keep up with the speed of the tapedrives.
I believe the speed for LTO4 should be near 120Mbyte/second. And probably
(can't find the source where I learned about it) the tapedrive
can slow down to about 50% to the minimum streaming speed.
Below the minimum streaming speed the drive needs to stop/restart
and while doing that, it looses capacity, as you can see from the
length numbers above. (And besides capacity loss, the wear/tear
on the tape and tapedrive becomes significant as well!)
So I would insert a tape in your drive, and run amtapetape on your
own configuration, to verify if your host is capable to drive the tape
at full speed. Also take into consideration that amtapetype does
not even read the data from disk. To be able to read the data at
at least 60Mbyte/sec from disk, you'll need a holdingdisk with a
striped raid setup.
--
Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525
Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, *
* quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., *
* stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, *
* ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, *
* Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... *
* ... Are you sure? ... YES ... Phew ... I'm out *
***
Re: strange twist on FAILED [data timeout] -- compress none not working correctly
On 2009-05-13 00:24, James D. Freels wrote: I got into debugging mode and discovered why this was happening. Apparently this upgrade, or a recent change in AMANDA, now forces (or by error) the data to be compressed with gzip before going from the client to the server. Even if I specify compress none (which I always have) or compress client none and compress server none, the gzip is still going on. I have never seen this before, and the gzip process is taking forever for my large filesystems. I can't seem to get it to stop. Note that the index is always gzipped, even when you using compress none. Did you came to this conclusion because there is a gzip process running? Or do you have indeed two gzip processes, and one (or both??) are slowing down everything? Or do you have a pathological filesystem where the name of the files is more data than the contents of the files? To hunt down what a client is doing I often use lsof on the gnutar process to find out what file it is currently reading. Maybe you got stuck on some unresponsive network filesystem (smb is frequently doing that -- I avoid mounting smb filesystem since I got burned badly). To get even more details you can strace the process as well. -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: newbie question about xinetd need restart all the time
On 2009-05-05 01:30, jing...@ucsd.edu wrote: I just tried install amanda from source on a centos 5.3 machine. Centos didn't have xinetd. I installed both xinetd and amanda. But amanda only work after immediate xinetd restart. After a while, it complains about network access problem. Thank you for your insight on how to stablize xinetd for amanda. I don't know where you got your CentOS from, but in mine the xinetd is installed by default. Even in the basic, minimal installs. And what are the complains about network access problems? Exact error messages do help to debug, for the time that my crystal ball is in the repair shop. -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: amanda wiki confusing me
On 2009-03-05 00:31, Deb Baddorf wrote: On this page http://wiki.zmanda.com/man/amanda.conf.5.html re the equations in the definition of flush-threshold-dumped int and flush-threshold-scheduled int and taperflush int What is the math symbol between t and d ? times (multiply) is the only thing that makes sense to me, but what I see (in 2 broswers; the third leaves it blank) is ? in a diamond box. IE I see h + s t ? d but I think it should meanh + s t (times) d yes it is a multiplication sign, but the conversion program that generated this html page from the source man page generated it in ISO-8859-1 codepage (char 0xD7) , while the webserver advertises it as UTF-8 encoded. Because these pages are generated by a program, you can't edit them in the wiki either. -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: best amanda tutorial/quickstart ?
On 2009-03-05 17:29, John G. Heim wrote: Any recommendations for the best amanda tutorial or quick start guide? Have already seen: http://wiki.zmanda.com/index.php/Main_Page and especially: http://wiki.zmanda.com/index.php/Quick_start -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: incremental backup
On 2009-01-07 01:30, aminukapon wrote: Does anyone know if amanda backs up using defs and how I can do incremental backups. Right now I can only get amanda to do regurlar full dumps. Can anyone help please? Yes, Amanda uses defs (if you mean that to be abbreviation of definitions and its broader variants like parameters etc.) What have you set as parameters now? Parameters that have influence on the schedule: - dumpcycle - runspercycle - record (in the dumptype of some DLE) Amanda will try to perform one full backup in one dumpcycle. The other runs in a dumpcycle will be incrementals. Did you ever read the wiki pages like: http://wiki.zmanda.com/index.php/Quick_start http://wiki.zmanda.com/index.php/Dumpcycle and even: http://wiki.zmanda.com/man/amanda.conf.5.html (which is also available as man amanda.conf on your host) -- Paul Bijnens, Xplanation Technology ServicesTel +32 16 397.511 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 http://www.xplanation.com/ *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: [Amanda-users] Scheduling backups
shantanu uab wrote: In summary: - Crontab will invoke amdump config-name - Amanda knows min. frequency of full backups needed and how many times amdump will be run from amanda.conf file - The clients/files to be backed up are specified in disklist (could be more than one) - Amanda ensures min. freq. of full backup is respected for all disklist entries, however not all backups taken during a particular amdump run would be of same type (full / incremental) Again, as mentioned in my last post, I am not sure whether dumpcycle is the min. frequency of full backup or is it the frequency of full backup? It is the frequency of when a full backup is done. It is not the minimum frequency. When needed, Amanda schedules a backup sooner than the dumpcyle parameter, e.g. to avoid doing a whole lot of full dumps on the same day in the future. And neither is it the absolute maximum frequency. And, it can also happen (but Amanda warns about this in the report) that a full backup is postponed beyond the dumpcycle frequency, e.g. when the amount of data would exceed the tape capacity. -- Paul
Re: Holding Disk Size
Matt Burkhardt wrote: I'm running out of holding disk space during the backup and I was wondering what I can do. Here's my scenario I've got a directory called Music that has 54GB on it. My disk has 36GB free. I have a second disk with 34GB free on it. Can I have multiple holding disks? yes. Example is in the amanda.conf that is distributed. If I give the disk list as each subdirectory under Music, will it need less holding space? Should I break up the backup jobs into several ones? Should I just buy a bigger disk? yes, yes and maybe yes (because they are cheap anyway). :-) For splitting large DLE's into smaller ones, see: http://wiki.zmanda.com/index.php/How_To:Split_DLEs_With_Exclude_Lists And, if it the server == client, then the advantage of using a holdingdisk is not that great, and you can do without a holdingdisk too (if your CPU is fast enough to feed the tapedevice at a fast rate when using software compression!). Just specify holdingdisk no in the dumptype of that DLE (or all of them). -- Paul
Re: [Amanda-users] Restoring amanda tapes without amanda
On 2008-11-27 22:49, rory_f wrote: As amanda uses tar, you surely can restore a tape (or a portion of a tape, perhaps just a dle?) using the command line 'tar' command, right ? Is it just the same as a normal extract ? tar -xvf /dev/nst0 ? Or do you have to do other things to ensure this works properly. Nothing is wrong with my amanda configuration, but It's something i want to document for my own reference :) For a lot of different scenario's, including how to restore without amanda software: http://wiki.zmanda.com/index.php/Restoring_files -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Huge file number in one directory makes problems while recovering
On 2008-11-21 15:38, Uwe Beger wrote: Hello list members, I am using amanda since many years successfully (and with still an old version VERSION=Amanda-2.4.4p3) for backing up a small server. The backup scheme is comp-user-tar for the disklist entry in question. I try to restore one subdirectory of this disklist entry containing about 2 MILLION files using amrecover. This restore takes very very very long and I am looking for a faster method. Any ideas are highly appreciated. Did you do in amrecover: cd subdirectory add * (this is very slow, and I even doubt it would work at all...) or did you do: cd parentfolder add subdirectory should be faster. But anyway, 2 million (200) files for one folder *is* much in any case. Even on disk, I wouldn't even *dare* to execute an ls there! And then, do you restore *over* the existing folder? In that case the OS will need to hunt through the existing *huge* folder removing eventually the already existing entry, and then probably slower each time, adding a new name into the directory. Anyway, a completely different approach would be to use amrestore and pipe to tar: cd /the/destination/top/level/dir amrestore -p /dev/nst0 hostname '^/disk/name$' | tar --numeric-owner -xpGvf - ./your/lost/subfolder (from: http://wiki.zmanda.com/index.php/Restoring_files#Using_amrestore ) And see the above page for e.g. the meaning of tar option G, possibly slowing down your process... -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: performance of backup process
On 2008-11-17 09:02, Mister Olli wrote: hi list, I'm running amanda for some time now, and it really rocks. It's a totally stable and easy to use backup solution with high reliability. great work.. since my data volume is increasing, I'm wondering how to speed up the backup process. My network includes about 20 clients (mostly *NIX) machines, and one amanda server which tapes to hard-drives. I'm wondering how to tune amanda in this scenario? are there any general hints and tips (beside tuning HD perfomance ;-)) The best program to spot bottlenecks and have an overview for the performance of the whole backup process is amplot. It takes some time to get it running, and study the output, but it's really worthwhile. right now I'm looking at the 'amstatus' of my daily backup job, which gives me the following output Using /var/log/amanda/daily/amdump from Mon Nov 17 08:31:51 CET 2008 172.31.2.10:/daten/business 011085m dumping 4380m ( 39.52%) (8:38:25) 172.31.2.10:/daten/company 1 16m finished (8:38:25) 172.31.2.10:/daten/intern 024001m wait for dumping 172.31.2.10:/daten/software 1 242107m wait for dumping 172.31.6.10:/daten/blub 0 1623m finished (8:40:33) 172.31.6.10:/daten/business 10m finished (8:37:14) 172.31.6.10:/daten/doku 0 5413m finished (8:45:19) SUMMARY part real estimated size size partition : 7 estimated : 7 284248m flush : 0 0m failed : 00m ( 0.00%) wait for dumping: 2 266108m ( 93.62%) dumping to tape : 00m ( 0.00%) dumping : 1 4380m 11085m ( 39.52%) ( 1.54%) dumped : 4 7054m 7054m (100.00%) ( 2.48%) wait for writing: 0 0m 0m ( 0.00%) ( 0.00%) wait to flush : 0 0m 0m (100.00%) ( 0.00%) writing to tape : 0 0m 0m ( 0.00%) ( 0.00%) failed to tape : 0 0m 0m ( 0.00%) ( 0.00%) taped : 4 7054m 7054m (100.00%) ( 2.48%) tape 1: 4 7054m 7054m ( 3.44%) ERNW-daily02 9 dumpers idle : client-constrained taper writing, tapeq: 0 network free kps:248976 holding space :408372m ( 96.12%) chunker0 busy : 0:00:00 ( 0.00%) chunker1 busy : 0:00:00 ( 0.00%) dumper0 busy : 0:05:25 ( 40.25%) dumper1 busy : 0:08:04 ( 60.00%) taper busy : 0:04:12 ( 31.21%) 0 dumpers busy : 0:05:23 ( 40.04%)not-idle: 0:05:23 (100.00%) 1 dumper busy : 0:03:36 ( 26.79%) client-constrained: 0:02:25 ( 67.29%) start-wait: 0:01:10 ( 32.71%) 2 dumpers busy : 0:04:27 ( 33.15%) client-constrained: 0:04:27 (100.00%) Above you say you have 20 clients, but this output shows only 1 client. is this a test. You can see that from amstatus that no dumper runs in parallel, because the default parameters have a restriction of only one dumper per client. When you have more clients, Amanda will run some dumpers in parallel, speeding up the whole process. from my understanding amanda is dumping (writing to holding disk) and taping (writing from holding disk to virtual tapes) at the same time. Indeed. Doesn't this reduce speed on the dumper caused by the head-seeks on the holding disc? Is there a way to prevent this scenario (as long as the holding disc is big enough for all data that belongs to the job)? Yes it does reduce the speed indeed. Optimizing the holdingdisk subsystem is indeed very important for the newer tapedrives like LTO4 that *need* a sustained feed at 80MB/sec *at least*, to avoid shoeshining. Things people often do: - use a separate controller for disk and tape. - use a raid striped over several disks as holdingdisk.M - use large buffers (tapebufs or device_output_buffer_size) - avoid reading from and writing to the holdingdisk at the same time (flush-threshold-dumped) (other tips are welcome) -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: [Amanda-users] amcheckdump output - questions :)
On 2008-11-04 17:45, rory_f wrote: Hi guys, any ideas?? My crystal ball is broken currently. It would help to spell out the question. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: [Amanda-users] amcheckdump output - questions :)
Validation process returned 2 (full status 512) Whats the easiest way to figure out what file(s) the error is refering to? And/Or is it really something to worry about. What about the last error, validation process returned 2? What does this mean? is there any way to run a check on the tape itself, and not the whole backup again, to save time? Thanks. +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +-- -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: [Amanda-users] amcheckdump output - questions :)
On 2008-11-05 17:53, rory_f wrote: Um, haha. its all on the same machine. i dont think thats the case. Im actually finishing up a backing up a 6tb project right now using tape_splitsize, and perhaps from now on i wont use it. we have a system in place to breakdown folder sizes for our whole file system here, so using just single tapes and doing root-tar whilst figuring out tape capacities ourself isnt actually out of the question. I'm not sure if I misunderstood you, just to be sure you understand me... These two concepts are orthogonal: - Amanda can use multiple tapes for one backup run. - Amanda can split one backup image over multiple tapes. It is only the second feature using the tape_splitsize parameter that makes restores more difficult. See: http://wiki.zmanda.com/index.php/How_To:Split_Dumps_Across_Tapes But instead of dumping one enourmous DLE in one backup image, you can use the include/exclude options with gnutar to split the DLE into smaller ones, each fitting on a tape. See: http://wiki.zmanda.com/index.php/How_To:Split_DLEs_With_Exclude_Lists Figuring out tapecapacity and optimizing tape usage is handled with the parameter taperalgo largestfit, as explained here: http://wiki.zmanda.com/index.php/How_To:Fill_tapes_to_100%25 -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: amfetchdump: running as user root instead of amanda ??
Jean-Francois Malouin wrote: * Jean-Louis Martineau [EMAIL PROTECTED] [20081104 13:46]: Jean-Francois Malouin wrote: Is this a new feature? I've done restore as root in the past I'm sure and never seen this before. Have been living blind all this time? :) So I guess I was lucky enough to do the amfetchdump in a dir owned by amanda and then it could recreate the dir structure and file ownerships... Maybe you were using amrestore? It doesn't need amanda privilege. I've used both for testing purposes before. I've just finished a new restore test on a new piece of hardware: # su amanda -c /opt/amanda/sbin/amfetchdump -b 2048k -p -d tape:/dev/nst0 top gustav /raid/ipl 20081104 | tar -xpGf - You've put the quotes too far. Put them only around the amfetchdump command, and pipe the result to tar, which still has root priviliges then: su amanda -c /opt/amanda/sbin/amfetchdump -b 2048k -p -d tape:/dev/nst0 top gustav /raid/ipl 20081104 | tar -xpGf - completes ok, looks good, the dle was successfully restored but tar didn't restore the original ownerships of the dirs and files, they all belong to user 'amanda' and group 'disk', its primary group, as I was suspicious it would do in the first place, but I wanted to be 100% sure before posting. Tar needs root privileges indeed to restore ownership. But amfetchdump needs to run as amanda. There could maybe a case for allowing to run as root, but that would open a whole lot of other problems, e.g. the debug directories like /tmp/amanda etc would be created with root ownership. That would give trouble for the next command, run as amanda, which would get permission to add its debug files to that directory. Besides, in general, it is safer and giving less chance to hit security problems when not running programs as root unless strictly necessary.
Re: [Amanda-users] amcheckdump output - questions :)
rory_f wrote: Paul Bijnens wrote: These two concepts are orthogonal: - Amanda can use multiple tapes for one backup run. - Amanda can split one backup image over multiple tapes. It is only the second feature using the tape_splitsize parameter that makes restores more difficult. See: http://wiki.zmanda.com/index.php/How_To:Split_Dumps_Across_Tapes But instead of dumping one enourmous DLE in one backup image, you can use the include/exclude options with gnutar to split the DLE into smaller ones, each fitting on a tape. See: http://wiki.zmanda.com/index.php/How_To:Split_DLEs_With_Exclude_Lists Figuring out tapecapacity and optimizing tape usage is handled with the parameter taperalgo largestfit, as explained here: http://wiki.zmanda.com/index.php/How_To:Fill_tapes_to_100%25 ok, smaller DLEs is fine. we have been doing this. i just thought using span was the only way for amanda to intelligently change tapes when a tape got full. we use a changer script. is that all we need to ensure it will change tapes when a tape gets full? for instance, we just want to make sure amanda will look at the size of the next dump, and move onto a new tape if needed. right? we use a holding disk also, which im sure makes it easier for amanda to know when to change tapes. ultimately we want to setup the right DLEs, and archive a whole proejct, around 6tb at at time. no interaction in this process is key. Indeed, use a changer and set runtapes to something greater than 1. Amanda will write a tape until she hits End Of Tape (signaled by a write error actually), and then move to a new tape. The whole backup image will be restarted on the next tape. (With using tape_splitsize you avoid starting the whole image, but start only the last chunk again) When using a holdingdisk, the dumps are collected in the holdingdisk unless the estimated backup image is larger than the holdingdisk. When one complete backup image is finished, Amanda start to put it on tape. Because many dumps can be collected simultaneously to holdingdisk, several of them may be waiting to be written to tape. If there are several to choose from, then the taperalgo setting decides which one will get tried first. A good setting is largestfit (see the wiki url above). If no backupimage that is ready will fit in the estimated available tapespace, then amanda will try the smallest, which has the largest chance of fitting in the unknown free space. The free space at the end of a tape is indeed unknown, because tapes do not always have exactly the same length; and when using software compression, the remaining space is even more uncertain, because it depends on the compressability of the previous tape images.
Re: [Amanda-users] Amrecover is not starting
On 2008-10-31 13:49, rakesh wrote: Hi Thanks a lot your recomendations worked, atlast amrecover has started but i haven't recovered any files till now.I would like to share some entries of the same , what am i missing in here ? Maybe you missed reading a manual? Maybe you missed learning some basic Unix skills? maybe you missed reading the errors messages and doing what they say? [EMAIL PROTECTED]:/backup/amanda_hold/DailySet1# amrecover AMRECOVER Version 2.5.1p1. Contacting server on localhost ... 220 BackUp-Host-Server AMANDA index server (2.5.1p1) ready. Setting restore date to today (2008-10-31) 200 Working date set to 2008-10-31. 200 Config set to DailySet1. 501 Host BackUp-Host-Server is not in your disklist. Trying host BackUp-Host-Server.translogicsys.com ... 501 Host BackUp-Host-Server.translogicsys.com is not in your disklist. Trying host BackUp-Host-Server ... 501 Host BackUp-Host-Server is not in your disklist. Use the sethost command to choose a host to recover amrecover What is the name of your host that you have configured in the disklist? ### [EMAIL PROTECTED]:/backup/amanda_hold/DailySet1# amrecover AMRECOVER Version 2.5.1p1. Contacting server on localhost ... 220 BackUp-Host-Server AMANDA index server (2.5.1p1) ready. Setting restore date to today (2008-10-31) 200 Working date set to 2008-10-31. 200 Config set to DailySet1. 501 No index records for host: BackUp-Host-Server. Have you enabled indexing? Trying host BackUp-Host-Server.translogicsys.com ... 501 Host BackUp-Host-Server.translogicsys.com is not in your disklist. Trying host BackUp-Host-Server ... 501 No index records for host: BackUp-Host-Server. Have you enabled indexing? Use the sethost command to choose a host to recover amrecover Can I expect some help over here PLEASE You can expect anything you like, but one fool can ask more questions than seven wise man can answer or something like that is probably also a good answer. (Not implying that I'm one of those seven.) -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: could not find a tape to use
On 2008-10-30 16:39, Timothy Ball wrote: I get this error when I try to run a amdump: http://pastebin.com/d1191d31e in the wiki I found this: http://wiki.zmanda.com/index.php/A_TAPE_ERROR_OCCURRED:_Could_not_find_a_tape_to_use and I don't think they're related, but since I'm new to amanda I'm still @ a loss as to how to debug this error. Any help would be really appreciated. Did you already amlabel any tapes ? Or do you have amlabel_new_tapes yes in the amanda.conf? What is the output amtape test-s3 show ? That configname seems like you're trying out Amazon S3 service. If yes, I guess you followed: http://wiki.zmanda.com/index.php/How_To:Backup_to_Amazon_S3 -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: [Amanda-users] Amrecover is not starting
On 2008-10-29 13:26, rakesh wrote:
[EMAIL PROTECTED]:/etc/xinetd.d# cat amanda
# default: on
# description: The amanda service
service amanda
{
only_from = BackUp-Host-Server BackUp-Client
BackUp-Client BackUp-Client BackUp-Client BackUp-Client
I guess the above line was edited to obscure the real names?
What if you temporarily comment it out, restart xinetd, and try again?
--
Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, *
* F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... Are you sure? ... YES ... Phew ... I'm out *
***
Re: [Amanda-users] 'Idiots Guide' for configuring Amanda on Linux?
On 2008-10-24 15:09, dceola wrote: And the rpm's force amanda into a different configuration when compared to what you would get from the (recommended) tarball install. im confused, through the reading i had done (with relation to installing amanda) i never saw anything that said anything other than to simply download one of the rpm packages from here: http://www.zmanda.com/download-amanda.php, since that is where the Quick start guide pointed me. So are we... Trying to help you, but not really knowing where to start explaining. From the quick start guide... -Run one of the installation packages available from Zmanda or elsewhere: --The Zmanda downloads site includes installation packages for many platforms. If you are using packages from Zmanda downloads page, go to the Zmanda packages quick start example. If you are using Windows client, refer to the Windows client page for installation instructions. --Many Linux and BSD distributions include Amanda packages, as does the Sun freeware site. -Compile Amanda from the source. Obtain the tarball from one of the sites mentioned above, or from the subversion tree I have been following the configuration instructions that are in the amanda Quick Start guide thus far. It seems that there are a few of you that seem to imply i am doing something wrong - which i obviously am - however i am simply following the quick start guide; so it's hard for me to figure out where i'm likely going wrong. So- at this point.. I'm thinking maybe i should simply uninstall the software and try to start over - assuming that's even possible, and i would assume it is... however, i had assumed that following the Amanda Quick Start guide would have been the best way to go, in that since it's from Amanda, it would lay out all that needs to be done. [Rolling Eyes] Actually, the Quick Start Guide is indeed the best place to follow. Maybe that Guide assumes that the user has a certain knowledge. When it says pick up the gun, aim and shoot, it assumes you're not aiming at your own foot. Also, that guide was written before there were RPM's and most people were installing Amanda by compiling from source, and manually editing all the config files from scratch. And indeed, the Guide says to add things here and there that are in fact already installed by the RPM's. Maybe we should a line indicating the section may be skipped when installed by an RPM. But it seems that you screwed up somehow as well. E.g. you somehow got an error message /usr/local/sbin/amdump not found. I never understood where you entered that pathname, resulting in the errormessage. Because if you just used the config files as installed by the RPM, then those do not refer to that pathname at all. Did you change config files? Were you mixing up -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: [Amanda-users] 'Idiots Guide' for configuring Amanda on Linux?
On 2008-10-23 14:19, dceola wrote: please excuse my lack of knowledge.. Ensure that the file is there. If it is, add /usr/local/sbin/ to the path of the user that is running amdump (in your case amandabackup). If not, then maybe you didn't compiled client AND server (without any special configure options, client and server are build). How do i do this? Also- i used the download from this page: http://www.zmanda.com/download-amanda.php that is listed for Fedora7 backup server. So, you installed an RPM? You installed the amanda-backup_server RPM? If yes, did that one install the program amdump in /usr/local/sbin ??? I was under the impression that that RPM installed it in /usr/sbin/amdump. Or do you still need to be enlightened on RPM's and program paths as well? Where do we need to start with the Idiot Guide :-) ? ls, find, rpm -ql, ./configure, patch ??? -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: [Amanda-users]
On 2008-10-23 16:34, dceola wrote: This is an edited version of a previous post haha, well i'm good at following instructions, typically ;) Then follow these instructions here: http://wiki.zmanda.com/index.php/Quick_start i definitely have yet to understand program paths (and rpm's i guess) here.. But maybe first learn some real basics first, so that you have can understand error messages, and better move around in Unix. We've all started like that once. (Even if 20 years ago.) This dates from 1986: http://groups.google.com/group/net.jokes/browse_thread/thread/60a9444026076c90/c49a0c0849fe8bb6 but needs changing some program names here and there to up to date in 2008. i downloaded the server RPM file that is listed for fedora7 on the amanda download page, and used the command /bin/rpm -ivh rpm name(s) when installing it (per the Installing amanda binaries page on the zmanda wiki). i have no idea where it installed amdump, i've just been following the instructions to install/config everything per the amanda qikc start guide.. so, past that i dont know... These commands will show which files the rpm installed: rpm -ql amanda-backup_server rpm -ql amanda-backup_client sorry for my lack of understanding on how everything works here.. its beginning to make me frustrated, i feel certain you guys are like geez look at this idiot or something.. [Shocked] (edit) i really hate not being able to figure this out on my own... so if you feel more inclined to point me towards any sort of reference material rather than trying to walk me through trying to figure this out (or even explaining this seemingly simple stuff), that works for me. i much prefer the method of teaching the guy to do it rather than doing it for him method myself. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Tape drives -- Recommendations?
On 2008-10-17 15:23, Greg Troxel wrote: Gene Heskett [EMAIL PROTECTED] writes: On Wednesday 15 October 2008, Dustin J. Mitchell wrote: On Wed, Oct 15, 2008 at 10:44 AM, Seann Clark [EMAIL PROTECTED] wrote: The other alternative I am looking into is getting a large external case and cramming it full of 1TB hard drives and using that as backup, but I would like a tape system that works well. This option is worth considering. HD's have many fewer moving parts than tapes/drives. Even if the critical failure rate is similar, the annoying failure rate of tape drives is much higher. Which is to say, they require a lot more fiddling. Dustin I'll back Dustin up on this one. Switching to a hard drive got rid of 99.9% of by backup problems. It Just Works(TM). -- Cheers, Gene There is certainly merit in the hard drive approach, but you can lose two critical properties if you aren't careful: full backups offline (not writable by computer) even while making next full backup. full backups taken to a remote location I solve these issues by doing backups to an external harddrive. USB-2 works reasonably fast, if the amount of data is not too large. I will be experimenting with eSATA real soon now. One of the ideas I have is to make a mirror with LVM of my vtapes to an external disk for offsite storage. I have been using LTO-2 for several years and have had little enough trouble, although I can't remember if it is very little or zero. Before that I had DDS-3 and that was occasionally annoying but not that bad. I am firmly in the tape camp at least for corporate use. But actually, I still use LTO-2 for offsite storage for our main office; it's only in 2 small offices abroad that I implemented backup to external USB disk. That disk gets exchanged on friday each week, and stored offsite. Using the USB subsystem makes the server (also small) rather unresponsive. Only workable in the weekends and nights. I hope eSATA will make better use of system resources. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: `include list' question
On 2008-10-13 13:55, Jukka Salmi wrote: Hello, while reading amanda.conf(5) I noticed the following comment about the `include' dumptype option: All include expressions are expanded by Amanda, concatenated in one file and passed to GNU-tar as a --files-from argument. They must start with ./ and contain no other /. I've been using include lists for a long time now, but some of them do contain lines with slashes after the starting `./'; this seems to work fine, though. So, why are slashes forbidden? Amanda needs to pass these strings to the gnutar option --files-from and gnutar does not do globbing expansion on those strings (as opposed to the --exclude option, where gnutar will do globbing). Therefor Amanda does the globbing before passing the result of the glob to gnutar. And for this globbing to work (and still be efficient), Amanda restricts itself to the toplevel directory (./some*thing) only. However, as boundary case, if the path contains more than one slash, then the loplevel globbing will not work, but Amanda will pass these strings unmodified to gnutar; and there you can get away with it. As long as you do not expect that pathnames having more than one slash will glob correctly, there is no problem. This dark corner of Amanda will probably change in future releases, so it's best to not rely on this too much. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Availability of Amanda Solaris10/Open Solaris package
On 2008-09-16 21:41, Martin.Hepworth wrote: Is the gnu project stow still around? This is/was really useful in splitting out stuff and bringing back to /usr/local. I'm still using graft for some packages, especially those where I need to be able to quickly fall back to a previous version, within a couple of seconds. (Like some additional packages on my mailserver and as a matter of fact, for those servers where I compile amanda myself instead of using an rpm, and on my Solaris machines). I switched from stow to graft long time ago, mainly because stows assumes it's the only owner of the http://www.gormand.com.au/peters/tools/graft/graft.html (there is also a description of similar systems in section, research) -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Is there a log that tells me the block size of a Backup Job
Doyle Collings wrote: [..] I have tarred to that drive at speeds of 5.5 gigabtyes a minute. A good method to find out how fast your tapedrive could work in your current environment is to write a scratch tape with the amtapetype command like: amtapetype -e 800g /dev/nst0 Takes a few hours. And this also a test to verify if you need to sacrifice more chickens or goats on the cables, adaptors and connectors, if you run into early EOT, usually by some tape write error.
Re: dumporder parameter
Johan Booysen wrote: Another question: How can I verify that those DLE entries were in fact only dumped after 05:00? In the Amanda email report I can see when the backups completed, and can more or less work out from that when writing to tape started for them. But I'd like to check when they actually started dumping. I checked through the logs in /var/log/amanda/server/daily, but the only relevant thing that may be what I'm looking for is from dumper.20080722235901.debug (my job started at 23:59, and I set up those disklist entries with starttime 0500): 1216790552.349935: dumper: getcmd: PORT-DUMP 00-6 11003 server2 feff9ffe07 /dir4/backups/backup1 NODEVICE 0 1970:1:1:0:0:0 GNUTAR X X X |;auth=BSD;compress-fast;index;exclude-list=/etc/amanda/exclude-list; OPTIONS features=9ffe00;hostname=server2; 1216790552.535954: dumper: name = 'server2' Any ideas on where I can find this information? Each line there is tagged with a timestamp, in seconds since epoch. To convert to human readable format use (assuming you are now in GMT+1 $ TZ=UTC perl -le 'print scalar localtime(1216790552.349935)' Wed Jul 23 05:22:32 2008 (leave out the TZ=UTC on the local computer to have the time in local time there). btw, the word PORT-DUMP indicates that this dump bypassed holdingdisk. If that is what you were expected, then that's ok. Bypassing holdingdisk and using a fast tapedrive usually does not work very well together: the backup datastream cannot follow the tape streaming, resulting in frequent stop/rewind/restart of the tapedrive. This really wears out the hardware of the tape (and at the same time looses lots of capacity in those gaps). Typically LTO drives can fall back to about half the top speed without shoeshining the tape; below that speed the process will damage the drive within a year for daily operations. Keep an eye on that too. -- Paul
Re: dumporder parameter
On 2008-07-22 10:37, Johan Booysen wrote:
Hi,
I got no reply so apologies if it was a stupid question. I'm now simply
using dumporder Ssss, and Amanda seems to be dumping disklist entries
starting from largest to smallest, which will work just fine for me.
I'm doing level 0 backups each night, and know how long the largest DLE
takes, so can reasonably accurately work out when to start the backup
job.
Correction:
dumporder Ssss
actually means that the first dumper will choose the the largest
dump available from the queue, the second to the fourth dumper will
chose the smallest from the queue.
If you have only 1 dumper running (inparallel 1, or without holdingdisk,
or client constrained) then, and only then the dumps go from large to small
with that parameter. Use to be more certain about the largest
to smallest.
My problem is that all the data to back up on Server2, Server3, and
Server4 isn't necessarily ready to be backed up until around 7am each
morning (this could possibly be moved to as early as 5am, but not by
much more than that). Server1 can be backed up at any time I wish
during the night. The sizes of the data to be backed up are fairly
static - I don't anticipate massive sudden increases (touch wood). I
also now have the capacity to force full backups each night. The data
on Servers 2, 3, and 4 will change completely each night anyway, so I
know I'll be backing up the amounts as above, even if I used Amanda the
classic way of level 0s and level 1s.
How can I configure the dumporder parameter to first back up Server1 (I
know how long a level 0 of that will take to dump) before doing the
other servers? Then I can plan more accurately when to actually start
the backup job.
Something like SSsS, if I want to back up Server1 first, then Server2,
then Server4, and then Server3 (Server3 takes the longest to create the
data to back up, so should come last)?
The dumpers are not spread in order of the disklist. You cannot map
the third letter to the third server.
There is the disklist parameter starttime:
starttime int
Default: none. Backups will not start until after this time of day. The
value should be hh*100+mm, e.g. 6:30PM (18:30) would be entered as 1830.
e.g.
Server2/dir4/backups/backup1 {
comp-user-tar
dumpcycle 0 # force a full backup each time
starttime 0700
}
Server3 /dir5/backups/backup2 {
comp-user-tar
dumpcycle 0
starttime 0705
}
--
Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, *
* F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... Are you sure? ... YES ... Phew ... I'm out *
***
Re: Another dumporder parameter question.
McGraw, Robert P wrote: When I backup I would like all my larger size files to be put on the first part of the tape and work down to the smaller size files toward the end of the tape. This I believe and hope will provide better tape usage. Yes, it does. If large size files are dumped last to tape there is a good chance that it will hit EOT and will then have to start over and a large chunk of tape will have been wasted and so will time. If smaller size files are sent at the end then if it does hit EOT not much tape is wasted. I have been using SS. So my question: 1) dose Amanda get the size of all the backup files and then figure the order that it will write to tape? Amanda does an estimate of each DLE, and from those figures decides on which dumps to choose first (dumporder). 2) will Amanda get the size of the tape and then figure the order that it will write files to tape. When taper is free to write the next dumpimage from holdingdisk to tape, then the taperalgo parameter decides which image from holdingdisk (only those that are completely done) is the next onej to write to tape. 3) will Amanda do the dumps in the order that it needs to write to tape. No. Dumporder is independent of tapeorder. But tapeorder is influenced by the availability of finished dumps in the holdingdisk. Dumps that bypass holdingdisk are always done last, even when, and mostly because, they are (too) large (to fit in holdingdisk space). See: http://wiki.zmanda.com/index.php/How_To:Fill_tapes_to_100%25
Re: software compression
McGraw, Robert P wrote:
I have decided to start useing software compression to see if I can get a
little better tape utilization.
I have set up the .conf file to use server best. I ran amdump with a test
DLE with one file. The backup system log that was email said every thing
went fine.
The problem is that I do not see any information about how well the software
compression did.
I have looked through the amdump logs and the logs in /tmp but still did not
see the information I was looking for.
What log files will give me the compression or at lest show me that it
worked.
From the last two mails you sent, it seems that you are missing
the report that Amanda sends after each run.
Did you get that mail (and read it)?
If not, you have to change a parameter in amanda.conf to indicate the
email addresses where you want the report to be mailed to; something like:
mailto [EMAIL PROTECTED] [EMAIL PROTECTED]
In that report that is sent after each run of amdump or amflush,
you will find the information like what DLE had some results missing:
Like things needing your attention:
FAILURE AND STRANGE DUMP SUMMARY:
wodan /RESULTS MISSING
wodan /bootRESULTS MISSING
wodan /var RESULTS MISSING
wodan /space RESULTS MISSING
planner: ERROR Request to wodan failed: timeout waiting for ACK
And general statistics:
STATISTICS:
Total Full Incr.
Estimate Time (hrs:min)0:03
Run Time (hrs:min) 4:39
Dump Time (hrs:min)7:31 4:01 3:30
Output Size (meg) 59425.724816.534609.2
Original Size (meg)258458.089127.4 169330.6
Avg Compressed Size (%)23.0 27.8 20.4
[...]
Look at these last three lines.
Output size: after software compression
Original size: before compression
And detailed statistics too:
DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-kB OUT-kB COMP% MMM:SSKB/s MMM:SSKB/s
-- - --
anubis/ 1166501088 6.5 0:3035.8 0:00 19220.6
anubis/boot 0108309280 85.7 0:01 12191.7 0:01 16644.2
anubis/space 1 258490 16608 6.4 0:32 518.1 0:00 40816.7
--
Paul
Re: How can I find what tapes are actually active?
On 2008-07-14 06:36, Bruce Thompson wrote: This seems like an obvious question, and yet I'm stumped figuring out how to do it. I am backing up four machines onto vtapes that I burn to DVD. Because the vast majority of my data being backed up never changes, this is almost effectively an archival setup (but not quite, and I'm not currently willing to set up different configs for the archival vs. non-archival, my setup works nicely for me). My runspercycle is fairly high since I don't mind accumulating incrementals over a fairly long period due to how little actual change occurs on a daily basis. Naturally, as disks are promoted to balance out the load and as normal timing comes around to run full backups, I end up with a number of vtapes that are no longer active. Looking at my tapelist, I see that all of the vtapes are currently listed as reuse and that there's no indication that tape 1 (for instance) is no longer active since the most recent full backup has effectively rendered it inactive. What I'd like to be able to do is have an automated way to tell which vtapes have been superseded meaning that all of the backup images have more recent images at that or lower level on a later tape. Is there an automated way to do this currently? I'm sure that I can devise a way to go through the index files are build a model of the current tape contents and note which ones are no longer active, but it feels like something that ought to have been done already. Has it? I believe you're looking for the command amoverview. What that command is missing, is to point at the last level 0 for each DLE, finding the oldest of that subset, and link that date to the tape. But that program is written in perl, so you can probably easily adapt it for that. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Index files and no-reuse
Marc Muehlfeld wrote: Hi, I have two sets. The first is for daily backup and the second is a level 0 dump that runs every first saturday of a month. The tapes of this set we store for one year offsite. After that time the tapes are overwritten. But the tapes from January and July were set to no-reuse and will never be overwritten (digital preservation). Are index files of tapes set to no-reuse removed after 12 month (tapecycle 12 tapes)? Or do I have to keep them somewhere else to have a comfortable recovering using amrecover? No, the index files are not erased after a dumpcycle. They are only erased when the tape is overwritten or when you do amrmtape.
Re: Why are there log files in /etc/amanda/DailySet1?
On 2008-07-11 08:55, Zanga Chimombo wrote: rant i have a problem with setting up amanda but more of that later. i have the following configurations: AMANDA_DBGDIR=/var/log/amanda CONFIG_DIR=/etc/amanda i have been looking in /var/log/amanda for clues to my problem to no avail. i have only just discovered log files in /etc/amanda/DailySet1/! e.g. log.20080711.0 and amdump.1 /rant They may have a substring in the name having the letters log, but during the history of amanda, they are used for much more then the initial purpose of logging only. http://wiki.zmanda.com/index.php/Log_Files -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Why are there log files in /etc/amanda/DailySet1?
On 2008-07-11 08:55, Zanga Chimombo wrote: my backup server is 192.168.0.1 and the client is 192.168.0.60. as a check, i am also backing up /etc on the server and that seems to go ahead without any problems. maybe my desired setup is unusual: all that i would like to do is a nightly backup of my home directory so that if i accidentally overwrite a file the next day i can recover it. i am not dumping to tape. i am trying to dump to a directory /tmp/amanda (tapedev=/tmp/amanda/DailySet1 in amanda.conf) on the server (as i said i'm only currently looking to recover the previous day's work if need be). i am not using a holding disk (i have commented the respective lines out in amanda.conf) Let's assume that you have your config correct; e.g. a line like tapedev=/tmp/... is certainly syntax error in amanda.conf, so let's assume you have something else there. (If not, post you have really in there.) Let's assume the command amcheck Dailyset1 did not return any errors. (If not, fix those errors first.) i have commented out tpchanger in amanda.conf and have a crontab entry amlabel -f DailySet1 DailySet1-00 the following line in amdump.X seems to give a clue: dumper: stream_client: connect to 192.168.0.60.46830 failed: No route to host why is the server trying to dump to the client? The dumper just tries to connect to the client GET the dump. Does simple networking between the client and host work? Depending on which version of amanda you use and which authorization protocol you configured the server and client need to connect to each other... More info, like version, protocol, and config is needed to help (and of course any firewall on the server or client). -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: amanda windows client times out
On 2008-07-10 04:45, FL wrote: Hi. I'm using the latest AMANDA 2.6.0p1 on a Debian Linux box (I compile the source--Debian packages are behind). The Linux clients work (except for an occasional 80 hour dump, as bad as that sounds on many levels, including dump levels). However, the windows client estimates time out. Not being a Windologian, I implore the group for guidance. I'm using bsdtcp, not ssh. My amstatus follows. FL [EMAIL PROTECTED] Daily]$ amstatus Daily Using /var/log/amanda/Daily/amdump From Wed Jul 9 18:05:02 EDT 2008 [...] curdata.gc.cuny.edu:c:0 10405m wait for dumping curdata.gc.cuny.edu:e:0 planner: [disk e:, all estimate timed out] curdata.gc.cuny.edu:e:Applications0 planner: [disk e:Applications, all estimate timed out] So you installed cygwin on those Windows clients, and run some port of Amanda-clients for Cygwin on Windows, right? And you're using gnutar for cygwin on those windows clients? Which version? For at least one windows client you did get an estimate for at least one DLE. Is there any difference in the definition of the DLE's of one that works or one that does not work? On the client, in the client debug files, (wherever those may be located in cygwin) do you see an error message when gnutar does it's estimates on the different drives? Or do they just take too long, and the amanda server has given up waiting for those numbers? -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Amanda 2.6.0 spanning
On 2008-06-25 14:48, Johan Booysen wrote: Well, that seems to work beautifully. I've forced a full dump of about 180GB, and amdump successfully spanned across the two tape drives. I'm doing a test restore now, but can already see that it works just fine. I'm still not 100% sure on how to perform restores if a disklist entry is spanned across two tapes, but one of the tape drives have died. Would I need to use amrestore instead of amrecover, and then manually untar all the restored chunks? In the parameters for the config that I added, a single DLE does not span two tapes, so you do not encounter that problem. You need to configure a dumptype for that having a parameter tape_splitsize and then only those DLE's having that dumptype will be split into chunks. See: http://wiki.zmanda.com/index.php/How_To:Split_Dumps_Across_Tapes I try to avoid that option if possible. Instead I try to fill my tapes using a technique that I explained here: http://wiki.zmanda.com/index.php/How_To:Fill_tapes_to_100%25 and, if you're using Amanda 2.6 or later, you can even add the newer parameters flush-threshold-dumped to achieve a better result. Thanks. Johan -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Johan Booysen Sent: 24 June 2008 13:39 To: amanda List Subject: RE: Amanda 2.6.0 spanning Hmm, that's weird. My tapecycle specifies 25 tapes. What I'm trying to do is to bring over the old server's index/log/tapelist/disklist files etc to the config on the new server, to test that I can do restores from tapes written by the old server. What I've done in the meantime is to create a second config to do a test amdump, so as not to affect the production config, so to speak. And I've labelled some spare tapes for the second config for this purpose so I don't mess with the production set of tapes. I must have missed something out regarding the existing tapes from the old server - will double-check it all again. Thanks! -Original Message- From: Paul Bijnens [mailto:[EMAIL PROTECTED] Sent: 24 June 2008 13:27 To: Johan Booysen Cc: amanda List Subject: Re: Amanda 2.6.0 spanning On 2008-06-24 13:49, Johan Booysen wrote: Hi, Looks like I'm getting somewhere this time. The server is happy with the two tape drives emulating a changer with 2 slots, and all amtape commands seem to complete ok. Just one question before I do a test run: Can I ignore the following warning about the second tape being still active and cannot be overwritten when doing an amcheck? No, you can't ignore that. slot 1:read label `daily-5', date `20080508'. .. .. slot 2:read label `daily-6', date `20080509'. Tape with label daily-6 is still active and cannot be overwriten. I'm assuming I can, since I double-checked that daily-6 is marked as reuse in the tapelist. The tapes are used by Amanda in an ordered rotation. The tapecycle parameter defines the size of that rotation. Amanda needs to be given the number specified in tapecycle before she accepts to overwrite that tape again. See man amanda.conf, tapecycle for a more detailed explanation. When you REALLY are sure you want Amanda to overwrite it, you can amrmtape followed by amlabel -f (specifying correct configs and labels). Or you can (temporarily) lower the count given by tapecycle in amanda.conf -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Amanda 2.6.0 spanning
On 2008-06-25 17:12, Johan Booysen wrote: Thanks for your replies. Wow - you guys confuse me! :) Ermmm...ok. Part of my problem is that I have one humongously huge DLE, and several smaller ones. And there has been some resistance to my suggestions that the one massive DLE be split up on disk. I'm now pretty much at a stage where very soon any full dump of this single DLE will require two tapes in itself. So I don't think I'll realistically be able to never have any DLE span two tapes... You can still split up a very large DLE using tar with include/exclude lists: http://wiki.zmanda.com/index.php/How_To:Split_DLEs_With_Exclude_Lists E.g. I have a DLE with thousands of subfolders. The folders are just numbers, and new folders get new numbers. I divided the DLE in 11 smaller DLE's depending on the last digit of the foldername (0-9 + one DLE with the rest). Even though Amanda can split a single DLE accross tapes, having smaller DLE's to backup still have other advantages: - Restores of single files are faster. The accidental deletes of a few files by endusers are much more frequent than disk crashes. (Amanda is growing features to speed the recovery of single files in very large images as well, but they are not yet ready.) - With smaller DLE's Amanda can spread the full dumps better accross the dumpcycle. Otherwise, when the full dump of that large DLE is due, the backup time takes too long, or could push out incremental dumps of other DLE's as well. - And last, smaller DLE's fit better on a tape avoiding splitting of a single DLE accross tapes. If one tape goes bad, the value of the following tapes becomes close to 0, especialy if you have compression enabled. And even then, I can understand there are still cases when one very large DLE need to be split accross tapes. But, if possible, I still try to avoid that situation. On the other hand, that won't really be a problem to me, unless I find myself in a situation where one tape drive dies and I can't do restores of DLEs split across multiple tapes. I'm hoping that, if I understand Dustin's reply correctly, that it will work. I'll switch off one tape drive tomorrow and test it. Summary: amrecover will prompt for a new tape (not by mail, but interactively) when you specify a device instead of a changer like: amrecover -d /dev/nst0 A few years ago, I did test that (and debugged and patched and patches got merged in), and it did work then. With amrecover_check_label Amanda would even verify if the correct tape was inserted, and prompt again if not. And, if my memory is correct, even a changer would loop through the slots, and, if not the correct tape found, would prompt as well. Feedback that it still works or not would be nice. Thanks very much. Wouldn't have been able to get this done without your kind advice. Johan -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dustin J. Mitchell Sent: 25 June 2008 15:35 To: Johan Booysen Cc: amanda List Subject: Re: Amanda 2.6.0 spanning If you give amrecover a specific tape drive to use, then it will prompt you to manually insert the correct tapes. If you configure chg-multi with only one tape device, though, then it will not prompt you -- it will just inform amrecover that it can't find the requested volume. So I'd recommend the first option. Dustin -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Amanda 2.6.0 spanning
On 2008-06-23 13:43, Johan Booysen wrote: Is there anyone who is actually using chg-manual for their Amanda setup, who would be willing to give me an example of their config files, such as amanda.conf and changer.conf (changerfile parameter)? Can't help with chg-manual, but... I avoid the use of chg-manual by having a large holdingdisk (4 disks 500GB each in raid5). I set maxdumpsize to a value large enough to dump what is needed in the run (800 Gbyte currently). For my archive run (full dumps only on weekends), I have two LTO2 tapedrives (using chg-multi to emulate a changer with 2 tapes), but the total backup volume now spans 5 tapes. The first two tapes are written during the normal dump during the weekend. On monday morning I insert the next two tapes, and run amflush. Somewhere early afternoon, I get a mail from amanda saying still some files are left on holdingdisk, so I insert the next tape and run amflush once again for last bits. Works good enough for me. And does not need manual intervention DURING a backup, like chg-manual does. I prefer having 2 tapedrives, instead of one (usually expensive) changer. That way i'm not completely blocked when a tapedrive needs repair. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Amanda 2.6.0 spanning
On 2008-06-23 15:49, Johan Booysen wrote:
That's very interesting. Not sure I completely understand your setup -
one normal config from Monday to Thursday, and another config doing a
level 0 only on Fridays?
Yes, I have two configs for amanda.
On Tuesday to Saterday at 0:04 early morning, I run the daily config.
That config uses vtapes on disk as backup using the chg-disk changer.
This is a classical run having fulls and incrementals mixed each day.
It includes all the disks except some large volumes (that contain
very static files only).
Using vtapes means there is no hassle to juggle tapes each day, and
restores are easy and fast as well.
On sunday morning, I run the archive config. This config does only
full dumps using LTO2 tapes. Those tapes are stored offsite.
I have 2 LTO2 tape drives, joined together with chg-multi to emulate
a changer with 2 tapes.
The config has more to dump than those 2 tapes, so I instruct amanda
to collect the rest to holdingdisk using the parameters maxdumpsize
and reserve.
Here are specialised settings in archive/amanda.conf:
dumpcycle 0 # results in mostly full dumps, if enough capacity
# to store them on tape and holdingdisk
runspersycle 1
tapecycle 5 tapes # I have many more, but allow overwriting fast
runtapes 2
tpchanger chg-multi
changerfile chg-multi.conf
tapetype LTO2
amrecover_do_fsf on
amrecover_check_label on
amrecover_changer changer
taperalgo largestfit # make better use of the tape capacity
reserve 0 # Important; default= 100% reserved for incrementals
maxdumpsize 800g # Important; default= runtapes times tapecapacity
define dumptype global { # included by every dumptype
program GNUTAR
index yes
record no # Important; do not confuse the daily config!
compress client fast
skip-incr yes # never do incrementals, even when out of space
strategy noinc # avoid calculating estimates for incrementals
}
For some slower DLE's, I even have estimate calcize and even
estimate server to avoid wasting time doing the estimates.
The file chg-multi.conf, in the same directory as amanda.conf contains:
multieject 0
firstslot 1
lastslot 2
gravity
needeject 0
ejectdelay 0
statefile /var/amanda/archive/changer-status
slot 1 /dev/nst0
slot 2 /dev/nst1
Anyway, I happen to have 2 identical tape drives, and the idea of using
both to emulate a changer with 2 tapes (like you do) sounds like a
good plan.
How did you set that up in amanda.conf (I'd imagine that I would have
/dev/nst0 and /dev/nst1 when both are installed)?
E.g.
runtapes (??)
tpchanger chg-multi
tapedev tape:/dev/nst0 (Specify 2 tape drives?)
changerfile /etc/amanda/daily/changer.conf (Need any parameters set in
this file?)
changerdev /dev/null (??)
See above (chg-multi ignores the changerdev parameter - you may omit it)
How do you handle restores? If one tape drive dies, and you have to
restore but Amanda thinks there should be 2 slots, how would you handle
that?
As you noticed I used the parameter amrecover_changer to give my
changer a name. Then I can do:
amrecover -d changer
(Actually, I made that the default builtin device)
And then amanda knows how to handle the two slots in my changer
all by itself.
When one tape drives dies, you can do two things.
Explicitly mention the device you want to use for recovery:
amrecover -d /dev/nst1
or (temporarily) change the settings in chg-multi-conf
to:
...
firstslot 1
lastslot 1# until the drive comes back from repair
...
# slot 1 /dev/nst0
# slot 2 /dev/nst1
slot 1 /dev/nst1 # drive nst0 is broken
Thank you.
Johan
-Original Message-
From: Paul Bijnens [mailto:[EMAIL PROTECTED]
Sent: 23 June 2008 13:21
To: Johan Booysen
Subject: Re: Amanda 2.6.0 spanning
On 2008-06-23 13:43, Johan Booysen wrote:
Is there anyone who is actually using chg-manual for their Amanda
setup,
who would be willing to give me an example of their config files, such
as amanda.conf and changer.conf (changerfile parameter)?
Can't help with chg-manual, but...
I avoid the use of chg-manual by having a large holdingdisk (4 disks
500GB each in raid5).
I set maxdumpsize to a value large enough to dump what is needed
in the run (800 Gbyte currently).
For my archive run (full dumps only on weekends), I have two LTO2
tapedrives (using chg-multi to emulate a changer with 2 tapes), but
the total backup volume now spans 5 tapes.
The first two tapes are written during the normal dump during
the weekend.
On monday morning I insert the next two tapes, and run amflush.
Somewhere early afternoon, I get a mail from amanda saying still
some files are left on holdingdisk, so I insert the next tape
and run amflush once again for last bits.
Works good enough for me.
And does not need manual intervention DURING a backup, like chg-manual
does.
I prefer having 2 tapedrives, instead
Re: Amanda 2.6.0 spanning
Following up on myself... On 2008-06-23 16:33, Paul Bijnens wrote: The file chg-multi.conf, in the same directory as amanda.conf contains: multieject 0 firstslot 1 lastslot 2 gravity Add a zero: gravity 0 But I guess you could find out that yourself... needeject 0 ejectdelay 0 statefile /var/amanda/archive/changer-status slot 1 /dev/nst0 slot 2 /dev/nst1 -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: lots of log files
On 2008-06-20 03:37, Paul Yeatman wrote: Do people routinely clean up the log directory? I'm up to about 82 gigs. I think it is time. Do I really care about log log.20060512.0 anymore? If you remove those files manually, than Amanda cannot locate older backups anymore with amadmin x find. When Amanda does not need them anymore, Amanda moves them to the subdir oldlog. And only from that oldlog directory you may remove them if you need the diskspace. I keep them to as data for historical evolution of my backups like backup sizes, times dumpspeed, tapespeed etc. http://wiki.zmanda.com/index.php/Amanda_log_files/Trace_Logs -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: amdump freezes
On 2008-05-25 18:55, jehan procaccia wrote:
hello,
some clients with big partitions (100Gbytes) freezes my amdump, I
usually get dumps errors which cannot end properly.
I have 2 questions,
1) how can I resolve that client error, timeout or whatever ?
This look suspiciously like the problem (and solution) described here:
http://wiki.zmanda.com/index.php/Mesg_read:_Connection_reset_by_peer
2) why the amanda server doesnt giveup after dtimeout (1800s) and
finishes its amdump ?
I think the intermediate % done messages that dump normally generates
are too far apart on the large dumps to keep the idle TCP connection in the
firewall open. Same problem can happen on the index TCP connection, when
dumping a few very big files.
Amanda does not give up, because the data itself is still
flowing through the data connection, not exceeding the dtimeout value.
However some firewall in between has closed the idle TCP connection carrying
the messages or index.
I use amanda-2.5.0p2-4 on a centos 5.1 server with disk (virtual tapes
on a raid 5) backup media.
here's the client error:
sendbackup: time 2185.293: 87: normal(|): DUMP: 4.10% done at 2642
kB/s, finished in 13:38
sendbackup: time 2485.300: 87: normal(|): DUMP: 4.67% done at 2632
kB/s, finished in 13:37
sendbackup: time 33385.302: 87: normal(|): DUMP: 60.34% done at 2453
kB/s, finished in 6:04
sendbackup: time 33685.307: 87: normal(|): DUMP: 60.89% done at 2453
kB/s, finished in 5:59
sendbackup: time 33753.599: index tee cannot write [Broken pipe]
sendbackup: time 33753.599: pid 25328 finish time Sun May 25 07:37:50 2008
sendbackup: time 33753.600: 109: normal(|):
sendbackup: time 33753.601: 112: strange(?): gzip: stdout: Broken pipe
sendbackup: time 33753.601: 112: strange(?): sendbackup: index tee
cannot write [Broken pipe]
sendbackup: time 33753.610: 87: normal(|): DUMP: Broken pipe
sendbackup: time 33753.611: 87: normal(|): DUMP: The ENTIRE dump is
aborted.
sendbackup: time 33753.611: error [compress returned 1, /sbin/dump
returned 3]
sendbackup: time 33753.611: pid 25325 finish time Sun May 25 07:37:50 2008
[EMAIL PROTECTED] /var/lib/amanda/int]
$ amstatus int --dumping
Using /var/lib/amanda/int/amdump.1 from sam mai 24 22:06:48 CEST 2008
helios:/home4 085174m dumping37083m
( 43.54%) (22:22:03)
helios:/home9 0 109374m dumping30169m
( 27.58%) (22:15:15)
amanda.conf:
etimeout -1200
dtimeout 1800
tpchanger chg-multi
define dumptype bi-comp-user-size {
comp-user
comment Non-root partitions on bi-proc machines
maxdumps 2
estimate calcsize
}
Disklist exemple of big partition
helios /home4 bi-comp-user-size
Thanks .
--
Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, *
* F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... Are you sure? ... YES ... Phew ... I'm out *
***
what is /etc/.gnupg ?
Instead of compiling from sources, as I did up to now, I tried the rpm from zmanda on a new host I'm preparing. Downloaded: amanda-backup_client-2.6.0p1-1.rhel5.i386.rpm When installing, I got these lines: ... May 26 2008 16:55:46: The directory '/var/lib/amanda/.gnupg' created successfully. May 26 2008 16:55:46: Ensuring correct permissions for '/etc/.gnupg'. May 26 2008 16:55:46: '/var/lib/amanda/.gnupg' Installation successful. ... The middle line, mentioning /etc/.gnupg, is, I suppose, not correct. Minor error, as it's only the message. The permissions of the directory '/var/lib/amanda/.gnupg' are correct after installing, AFAIK. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: [Amanda-users] Need help setting up Amanda on Sles10
On 2008-05-21 15:25, bperrotta wrote: When Installing go message couldn't create Amanda user. Using root user instead. Manually created Amanda user later on but don't know which groups to add it to. Got errors saying home directory was not created had other trouble with user rights. No sure what to do at this point help appreciated , used the suse sles10 rpm and the guide to install in less time than ordering a Pizza Then came the errors. Not a Linux guru this is mostly trial and error. trying to backup to a travan drive, a Novell oes Data volume directly to the usb travan drive on the server, which is already detected by Yast. Using Amanda because of budget constaits in buying a commercial product. directions hard to understand lost at this point. Help appreciated, The wiki is a great source of help: Start here: http://wiki.zmanda.com/index.php/Quick_start Then ask more detailed questions on the list if anything is unclear. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: /var/lib/amanda/gnutar-lists : huge !
On 2008-05-07 17:44, FM wrote: Hello, Is it safe to do some clean up in this folder ? No. It's the list of statefiles, which gnutar uses to decide if a file should be in the incremental backup. Erasing them (or getting them out of sync) will result in incrementals be much larger than normal. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Out of space problem
On 2008-05-06 02:30, Nigel Allen wrote:
I'm experiencing an odd problem with a USB DAT drive that keeps running
out of space. Apologies for the length of the post.
The drive is supposed to be 36 / 72 GB.
Ok. The meaning of those numbers is:
- around 36 gigabytes (in powers of 1000 not in powers of 1024) space
to put the bytes on.
- in the optimistic premise that the internal compressor can achieve
a reduction by 2, you can feed 72 GB data, which will fit in the
36 GB raw space on the tape.
You must understand that the compression is an algorithm, just like
gzip or bzip2 (actually a less optimised version of the older compress
program). The tape drive does NOT double the capacity by writing
the bits in higher density, or with double number of tracks or so.
In real life this means that those tapes can hold about 33 Gibyte
(in base 2^10=1024). This is the number that Amanda needs.
See http://en.wikipedia.org/wiki/SI_prefix#Computing for the difference
between GB and GiB. (Amanda uses the MiB, GiB, but writes it as MB, GB, etc)
Then also realize that the compressor built into the those DAT drives
actually EXPANDS data that is not compressable by about 20%. (Contrary
to the compressor built into the more modern LTO drives, which do not do
that.)
When feeding the tape drive with already compressed data (you use bzip2
already), and you use hardware compression, then you loose again
tapespace.
Here's the kind of thing I see when I run a level 0 dump.
These dumps were to tape DailySet1-18.
*** A TAPE ERROR OCCURRED: [No more writable valid tape found].
[...]
taper: tape DailySet1-18 kb 31022240 fm 2 writing file: No space
left on device
driver: Taper error: [writing file: No space left on device]
The tape drive hit EOT after 1022240 KiB = 29.5 GiB.
Therefore I believe you have both hardware + software compression
enabled, resulting in a loss of about 3 GiB of space.
[...]
define tapetype HP-DAT72 {
comment HP DAT72 USB with hardware compression on
length 72 G
}
Especially when using software compression, use the native
capacity of the tape in the definition: Try with 34 GiB.
Or better, use amtapetape -e 35g /dev/yourtapedrive to get
an estimate (takes about 3-5 hours normally).
[...]
define dumptype custom-compress {
global
program GNUTAR
comment Dump with custom client compression
exclude list /etc/amanda/exclude.gtar
compress client custom
client_custom_compress /usr/bin/bzip2
}
When using bzip2, the hardware compressor can never compete with
that (except in speed). So better disable hardware compression.
mail.airsolutions.com.aumapper/VolGroup00-LogVol00
custom-compress
mail.airsolutions.com.ausda1custom-compress
When using gnutar, why do you insist on specifying a device name.
Why not use the mount point itself. (Amanda will do the conversion
for you, but why make it so obscure.)
Any idea where I can start would be appreciated (apart from bigger
tape or less data).
Another improvement is to break up the large dataset into
smaller ones, so that each one dump is much smaller than a the
capacity of your tape. That way, Amanda can better spread the dumps
over different tapes/days. See:
http://wiki.zmanda.com/index.php/How_To:Split_DLEs_With_Exclude_Lists
--
Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, *
* F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... Are you sure? ... YES ... Phew ... I'm out *
***
Re: Can't seem to disable software compression.
On 2008-04-30 13:45, Edi Šuc wrote: Hi. It seems that I can't disable software compression in amanda. I tried everything that could in my opinion have an impact on compression. But no luck. The problem is that i have a relatively slow machine and it takes forever to complete the backup. If I check the processes i see this: 19021 ?S 0:00 \_ /bin/sh /usr/sbin/amdump AAA1 19031 ?S 0:00 \_ /usr/lib/amanda/driver AAA1 19032 ?S 0:01 \_ taper AAA1 19037 ?S 0:01 | \_ taper AAA1 19033 ?S 0:08 \_ dumper0 AAA1 19034 ?S 5:31 \_ dumper1 AAA1 19074 ?S 0:08 | \_ /bin/gzip --best 19035 ?S 0:00 \_ dumper2 AAA1 19036 ?S 0:00 \_ dumper3 AAA1 19066 ?S 8:59 \_ chunker1 AAA1 I suppose that there shouldn't be a gzip --best if I disable the compression. I also tried to use the directive compress fast in the define dumptype section, but no luck. gzip --best is still there (it should at least change to gzip --fast). The index (list of filenames in the backup) is ALWAYS compressed using gzip --best, and I bet that disabling the index (or not compressing it) will not speed up the process. The bottleneck you have is probably something else. client-disks? network? I can only say that usually the power for the amanda server is not that important (except for those modern tapedrives, that expect to be provided with a continued 80 Mbyte/sec or more, which simple IDE drives cannot provide). I had for a long time an Amanda server having 128 Mbyte RAM, 300 Mhz Celeron, 80 Gbyte IDE holdingdisk and AIT-1 tapes (35 Gbyte native). It was serving 120+ DLE's from about 30 hosts. In that time the bottleneck was on the clients (disk-IO + software compression). And for my home config I have a similar old computer 500 MHz, 192Mbyte RAM but with twice software raid-1 250GB disks for vtapes + one 60 Gbyte for root filesystem, software and holdingdisk. And still the bottleneck is the client and the network (wireless is not that fast; and powerline is not fast either). To locate bottlenecks, use the commands amstatus and amplot (especially the last one, is very enlighting!). In your config I see: netusage 600 Kbps I would eliminate it, as well as the limits on each interface you specified. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: disk /, all estimate failed
On 2008-04-14 09:39, terry wrote: This is in the logs FAIL planner bigfish.bluelight.org.uk / 20080414 0 [disk /, all estimate failed] a clue to go on I will go see if I can google it bigfish# tail /usr/adm/amanda/log.20080414.0 [...] bigfish# tail /usr/adm/amanda/amdump.1 [...] These are the server logs, but I want the client logs under the AMANDA_DBG directory See: http://wiki.zmanda.com/index.php/Amanda_log_files/Debug_Logs -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: disk /, all estimate failed
terry schreef: I followed this howto here http://www.howtoforge.com/disk_based_backups_amanda_debian_etch and of course adjusted for my os but when I run a backup I get an email containing These dumps were to tape DailySet1-2. The next tape Amanda expects to use is: DailySet1-3. FAILURE AND STRANGE DUMP SUMMARY: bigfish.bluelight.org.uk / lev 0 FAILED [disk /, all estimate failed] [...] any pointers welcomes Just pointers: a more complete setup for the beginners: http://wiki.zmanda.com/index.php/User_documentation Frequent problems and their solution: http://wiki.zmanda.com/index.php/Troubleshooting To be able to help you, you have to give more information and especially the error messages found in the debug files that amanda generates for each run. One of the first things to try is running amcheck and solve every problem it encounters. Did you run amcheck without errors/warnings?
Re: amtapetype_problem
On 2008-03-13 14:51, Dustin J. Mitchell wrote: On Thu, Mar 13, 2008 at 5:42 AM, Javier Guerrero [EMAIL PROTECTED] wrote: My version of amanda is amanda-backup_server-2.5.2p1-1.sles9.i586.rpm , how i could see if this patch is in my installation of amanda? I've tried to find with locate command any file similar to tapetype.c, I suppose that I would have to download a source version, but, which are the steps to apply the patch later? You're correct -- you would need to recompile from source to apply the patch. However, the patch is only for amtapetype, which is only responsible for determining the tapetype you would use in your amanda.conf. If other folks on the list have a working tapetype declaration for your drive (HP Ultrium LTO1(100/200 GB)), then you may be able to skip using amtapetype altogether. If that doesn't work, I can build a patched version of that utility and send it to you by email. Dustin (Helpful as always, Dustin!) But I think the problem is deeper: if it is indeed that on this machine rewind flags success too early, or that subsequent requests do not block until the rewind is completely finished, then I believe amlabel will have the same problem: it writes the label, rewinds, and verifies if the label was written correctly. And maybe amdump also has problems, because that one reads the tapelabel rewinds, and writes a new tapelabel, followed by the backup images. Especially in amdump the Design of Amanda is that the tapedrive is opened and closed only once for the whole duration of writing to tape. Only that way Amanda can guaruantee that no event on the tapedrive gets unnoticed (someone replacing the tape between the close and the next open, or a scsi bus reset, which rewinds the tape all by itself, etc.) As happed to this Other Software here: http://tinyurl.com/25caga http://tinyurl.com/2enysj (makes me glad I'm using Amanda). It would help if Javier would give some details on the OS he is using. Does the sles9 in his rpm name suggests Suse Linux Enterprise Server 9? Or did just use that rpm and forced it to install on his OS? If SLES9, is anyone else seeing that problem (easy to test with amtapetype)? (Joseph had the problem on an Alpha system). IMHO I think the driver is broken, not Amanda. Rewind should return only when the tape *was* rewound... -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: amtapetype_problem
On 2008-03-12 14:56, Jon LaBadie wrote:
On Wed, Mar 12, 2008 at 01:51:05AM -0700, Javier Guerrero wrote:
I have a problem to determine the type of my tape, I know that is an HP
Ultrium LTO1(100/200 GB).
I want to know the specific parameters to insert into amanda.conf, because
the actual parametres give an error(the same error of amtapetype) when I run
amdump and the backup is big enough.
[EMAIL PROTECTED]:~ amtapetype -o -e 100g -f /dev/nst0
Estimate phase 1...
Writing 2048 Mbyte compresseable data: 36 sec
Estimate phase 2...
amtapetype: could not write any data in this pass: Input/output error
The content of my amanda.conf is:
define tapetype HP-LTO1 {
comment
length 101376 mbytes
filemark 0 kbytes
speed 13872 kps
}
First, step back from amanda.
Can you use your tape drive without amanda.
mt with various cmd arguments
dd several GB to and from
tar to and from
I seem to remember there was indeed a problem on some hosts
where rewind flagged success and finished, but the tape drive
was actually not ready to accept new requests, resulting in IO errors
until the tape was fully rewound.
But I do not remember anymore if or how it can be solved
(except by modifying the code and waiting longer after the rewind).
--
Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, *
* F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... Are you sure? ... YES ... Phew ... I'm out *
***
Re: Tapelist
On 2008-03-11 17:57, Arthur Smith wrote: Thanks! I want to be sure I understand correctly: I have 10 tapes, labeled monday1, tuesday1, etc.. through friday2. I should now be able to recreate the tapelist file like so: 0 monday1 reuse 0 tuesday1 reuse and so on, and amanda will use them in the order they are put in, correct? As if they were actually new tapes (even though they're not)... Am I on track? You are on track about the format of the tapelist file but falling off on the other side of the track when choosing the tape labels. :-) It's not a good idea to label the tapes monday etc, unless you are 100% sure that backup on each day succeed, and you never take a day off or forget to switch tapes. The first time the backups from e.g. monday are not on that tape the tapes get out of sequence. That can happen because you forgot it insert it, or because the server crashed, or you need that tape to flush the holdingdisk on the previous day, or you need to keep one tape aside because it contains a full backup of a server that crashed, and you're waiting for the new disks to arrive, etc. (And I probably forgot another 100 reasons...) Just number them Daily-01 etc. My 0.02€ advice. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Backing up VMware-VMs
On 2008-03-06 21:38, Steve Wray wrote: Stefan G. Weichinger wrote: Patrick M. Hausen schrieb: So if you want to backup/copy an entire VM with the guarantee of consistent hard disk state, you need to shut it down. Copying a multi gigabyte virtal disk file is bound to take quite some time. So you need to power down your virtual machine for what can amount to hours. What about combining it with LVM-snapshots? Are LVM snapshots working properly now? The last few times I tried it (several months ago) it was a total disaster and not something I'd want to risk even attempting on a virtual machine. What kind of disaster on which kernel? I had problems when using LVM2 in CentOS/RHEL 4.2 or so, but in 4.5 and 5 it is stable as long as you have 256 Mbyte RAM or more (I have one machine with 128 Mbyte RAM running CentOS 4.5, and that one had a kernel panic once every 6 months or so due to LVM2 snapshots -- I do not use snapshots anymore on that machine since last time it crashed about 3 weeks ago. Most of my other servers take snapshots twice a day (as cheap backup during noon, and as quiet filesystem for nightly Amanda backups), and have no problems with it. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: tar returned 2
On 2008-02-27 15:56, Rodrigo Ventura wrote: Hello, I'm getting sporadically errors like this: FAILURE AND STRANGE DUMP SUMMARY: omni/ lev 1 FAILED [dump (30447) /usr/bin/tar returned 2] and I don't have a clue on what's going on. I grepped /tmp/amanda and found no reference to this. Any clues? A little bit further in the same mail message is is a section, named: FAILED AND STRANGE DUMP DETAILS: It should have a section showing the output of the complete tar session for that DLE. Lines beginning with a '|' are expected. You should focus on the lines beginning with a '?': those are the ones that Amanda did not expect, or that are error messages. If there is no information, then you better go to the client itself and look in the /tmp/amanda (AND subdirectories) to find out what went wrong. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Best Amanda Setup for Restores
Gordon J. Mills III wrote: Is this simply a case of not realizing that 'amrestore' is generally used for complete dle restoration while 'amrecover' is used for individual file/directory recovery? I am beginning to question my sanity here G. I am using the amrecover interface to recover it but it seems that it will first extract the entire tar'd up dle to /tmp/Amanda before extracting the small folder that I requested. Am I wrong about that? Let me assure, AFAIK Amanda does not do that. How did you come to that conclusion? What is the name of the file that is created in /tmp/Amanda (uppercase A in the name???) If this is the case (and it seems to be what I have experienced) then my problem is that the entire tar'd up dle will not fit on the partition which contains /tmp/Amanda/. So to get at this particular folder for this restore I need to change that default to another drive which does have enough free space. To further clarify, I am running Debian and using the packages in the debian archive for Amanda. I am using vtapes also. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Tape Length Problems
On 2008-02-14 04:16, Nigel Allen wrote: Hi I have a client running FC6 with an HP StorageWorks DAT72 USB internal tape drive which according to the documentation can do 36G native and up to 72G compressed. We are trying to use Amanda to just back up the server so it is configured as both server and client. We ran amtapetype on it which suggested a tape size of around 32GB. [...] FAILURE AND STRANGE DUMP SUMMARY: mail..com.au mapper/VolGroup00-LogVol00 lev 0 FAILED [dump larger than available tape space, 53330360 KB, but cannot incremental dump new disk] Indeed 53 Gbyte backup image does not fit in 32 Gbyte tape. The disklist looks like this: mail..com.aumapper/VolGroup00-LogVol00 high-tar mail..com.ausda1high-tar (sidenote: when using tar, why do you use device names? not not directory names?) and current disk usage is: FilesystemSize Used Avail Use% Mounted on /dev/mapper/VolGroup00-LogVol00 225G 52G 162G 25% / /dev/sda1 99M 11M 83M 12% /boot tmpfs 252M 0 252M 0% /dev/shm I'm obviously missing something here - I understood that amanda would take the compression into consideration. Should I be using software compression or hardware? If software, how do I turn of compression on the tape device? I think you missed to configure the correct tapetype definition to 72 Gbyte in the experiment. Find if Amanda thinks the same about the config you changed: amgetconf DailySet1 tapetype note the output: this is the tapetype that Amanda will use; let's assume the output was DAT72; then try: amgetconf DailySet1 tapetype:DAT72:length And Amanda shows the output in Kilobytes. But the real solution: If you have the CPU-power, then software compression is surely better than hardware compression when using DAT72 tapes. But, make sure you have the hardware compression disabled, otherwise you actually loose 20% capacity by the dumb hw-comp algorithm on those tapedrives that expands uncompressable data. To disable hardware compression, see: http://wiki.zmanda.com/index.php/Hardware_compression When estimating, Amanda makes a dummy backup, and notices the output of tar indicating the total bytes output. For compression, it assumes a ratio learned from the previous dumps. When no history exists, Amanda assumes a ratio, that you configure in amanda.conf with the comprate parameter, which defaults to 0.50 (= compression to half). If this is indeed the first backup ever of that DLE (as the message says ...cannot incremental dump new disk), then you have to figure out why Amanda thinks 53 Gbyte * comprate is more than 32 GByte. Look carefully in the parameters like comprate from: amadmin DailySet1 disklist mail..com.au mapper/VolGroup00-LogVol00 For DLE's that had already some backups run, you can find out the historical data that Amanda uses to compute the comprate with: amadmin DailySet1 info somehost /some/disk -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: amreport takes long to complete - BogusMonth
On 2008-02-14 09:56, Theodotos Andreou wrote: Since a month ago I have the following problem appearing: When amdump is completed amreport takes long to complete. When it finally finishes it won't send any mail. Running amreport manually results in a mail with subject: It takes so long because the dump file contains lots of strange lines probably. Strange = not expected in the normal output of tar: Amanda collects those messages, prepends a '?' to it and displays them in the mailreport. Amanda then leaves it up to you to decide if that is bad, or harmless. Because there are so many lines, generating the report takes very long. And probably when sending the mail, it is refused by the mailserver because the message is too large. If it happens to be a full dump with many files, then the mail was too large; when it contains only an incremental dump, it is not that large, and you get the report. Org DailyJob AMANDA MAIL REPORT FOR BogusMonth 0, 0 and body: *** THE DUMPS DID NOT FINISH PROPERLY! even though amstatus reports that all is OK. I even verified that the images are present with amrestore. Yes, but are all the images intact, and do they contain all the data that was on the disk? Some DLE at least flagged as not good enough. Or maybe the filesystem containing the amdump.XX log file was full resulting in amreport not finding the normal log lines indicating a good backup. It is also peculiar that the problem is intermittent, It happens most of the time but not always. I was not able to figure out what triggers it. I am using amanda 2.5.0p2 on CentOS 5. I tried to strace on the running amreport and I got some strange results: read(3, Warning: Cannot fgetfilecon: No data available\n ? gtar: ./ionc/environment/3.4.old/kreatel/adk/mipsel-kreatel-linux-gnu/mozilla/include/uriloader/nsDocLoader.h: Warning: Cannot fgetfilecon: No data available\n ? gtar: Is this maybe related to: https://bugzilla.redhat.com/431879 Did you recently upgraded tar, which generates a warning for each file where se_linux context is not defined? -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: mixing revs
Jeremy Mordkoff wrote: I'm running amanda 2.4.4 on my server one of my client systems got upgraded to 2.5.1 and now I can't back it up. The client seems happy, but the server logs a timeout on sendsize. They should be able work together. Is there a workaround besides downgrading amanda on this one client? Some stuff has changed in the 2.5.1 setup, especially in the (x)inetd configuration. http://wiki.zmanda.com/index.php/Configuring_bsd/bsdudp/bsdtcp_authentication Notice the server_args. Without it, it will not work. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: NFS mount tar incremental problem
On 2008-01-25 02:08, Jordan Desroches wrote: To answer a portion of my own question, the linux command (for the NFS mount point /mnt/thayer/home): $stat --format=%d /mnt/thayerfs/home will give the decimal device number necessary to run the tar-snapshot-edit script. It still doesn't answer the more puzzling question of why tar is not picking up mount as NFS as the documentation says it should. The idea to use the device number to identify the device is wrong. Quoting Linus himself: The device number is a random cookie, not a unique identifier. http://lwn.net/Articles/65195/ It used to be that the device number was a static number (e.g. something like device number = major*256+minor, in the days when major and minor numbers where hardcoded in the kernel). The right way currently is to not consider the device number to unique identify a system. It is only unique among the currently other device numbers present on the system, but the same device when unmounted/remounted is not guaranteed to get the same device number again. That is why udev was invented, and there you can use some other property of the device to get a static name: http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev_vs_devfs Gnutar still relies on the fact that the device number is static and does never change, not even after a reconfiguration of the devices (e.g. a reboot, or remove/reattach of a device). Best, Jordan On Jan 23, 2008, at 9:29 PM, Jordan Desroches wrote: I've dug further into the gnutar-lists directory, and I think I know what is causing the problem, but I don't quite know what to do about it. I have a NFS mounted directory /mnt/thayerfs/home Here is a section of the incremental file from 1/19/08: [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]/chen.. here from 1/20/08: [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]/gre. here from 1/21/08: [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]/sp here from the 1/22/08: [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]/oli and here, from 1/23/08: [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]/st. The first thing that raises my concern is the leading 0. I believe, according the tar docs, that indicates that tar is NOT detecting the mount as a NFS mounted partition. If it had detected it as an NFS partition (which is would do because apparently tar takes different action with NFS paritions), there would be a leading 1. The second thing is that that the device number changes between the 22nd and the 23rd from 23 to 22. There was a reboot between those days. Is there a way of preventing the device number from changing? If not, then is the knowledge of the device number enough to run the script Paul suggested? If running the script is the solution (and to ask a potentially, and I hope simple question), how do I determine the device number of a NFS mounted partition to tell the script to change it? Thanks for your help :-) Jordan On Jan 23, 2008, at 11:19 AM, Paul Bijnens wrote: Jordan Desroches wrote: Hello all, I've been having a problem with incremental dumps on a NFS mounted Netapp. AMANDA runs great until I reboot the client (or remount the NFS shares on the client). At that point, while calcsize predicts what I believe is the correct incremental dump size, tar proceeds to do a full dump of all the NFS mounted files. I believe this has to due with something changing between mounts that tar is translating as a change to all files. Upon reading some of the documentation for tar, it indicated that in the incremental dump gnutar-lists, there should be a 1 preceding every entry to indicate that the file is NFS mounted because (Quoting http://www.gnu.org/software/automake/manual/tar/Incremental-Dumps.html;): Metadata stored in snapshot files include device numbers, which, obviously is supposed to be a non-volatile value. However, it turns out that NFS devices have undependable values when an automounter gets in the picture. This can lead to a great deal of spurious redumping in incremental dumps, so it is somewhat useless to compare two NFS devices numbers over time. The solution implemented currently is to considers all NFS devices as being equal when it comes to comparing directories; this is fairly gross, but there does not seem to be a better way to go. Here is an example from one of my gnutar-lists, showing what I believe are preceding zeroes, indicating that tar thinks that the files are not on NFS: [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]/unclaimed_afs/nmlhome/mcbride/.desktop-nauset.dartmouth.edu/[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@./spacescience/web/wl/per/[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED
Re: selfcheck request timeout
On 2008-01-25 12:05, Marc Muehlfeld wrote: Paul Bijnens schrieb: I would expect to see amandad listen on ALL interfaces, especially including localhost!! I meant loopback instead of localhost... but you got the idea. Verify xinetd configuration for amandad. I think it should be no problem. Amandad is just bind to the interfaces it have to, in my configuration. I think localhost isn't required if server/client are connecting through FQDN and not localhost. That depends on /etc/hosts. His client is his server as well. And with an entry in /etc/hosts like: 127.0.0.1myhost myhost.example.com localhost localhost.localdomain (and nsswitch.conf listing files before dns for the hosts), then you better be listening on the loopback interface too! I find it really strange that amanda binds only the the external interface, and not to loopback. Why??? -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: selfcheck request timeout
On 2008-01-23 04:40, Kermit Short wrote: Hey gurus! I've checked many many resources about this error and I've done just about everything they've asked, but I can't get rid of it, and I don't really even know if it's a show stopper or not. Here's my situation: [...] backup set name. In installing/configuring the client, I've done the following: customized the /etc/amandahosts (symlinked by ~backup/.amandahosts) especially here: are the permissions of this file correct? Is that path up top ~backup/.amandahosts secure (not writeable by anone but root or backup-user?) customized the /etc/hosts.allow for the amandad, amindexd, and amidxtaped daemons customized the /etc/services to register the 3 above daemons as services for the xinetd superserver created a drop file in /etc/xinetd.d with the standard recommended parameters and restarted the xinetd superserver OK. we have to believe you that you did not make any error in there. We cannot verify that part of the setup. something that wasn't mentioned on any of the FAQs and mail lists was to change the group on the 3 above daemons to the backup group (the group the 3 above daemons??? which ones? That also seems to imply that you have a discrepancy between the username that is compiled into the executable ( with the configure option --with-user=...) and the one that you use to run the set of programs. They should match! What is the output of amadmin xx config and look at the expected username, in the configure options. specified for the backup user which will be the user running amanda) IPTABLES is not running on this host, so it isn't a firewall issue, unless the port calls go running about the LAN rather than staying internal, which I can't imagine is happening. netstat -a |grep amanda reports the following: tcp0 0 *:amandaidx *:* LISTEN udp0 0 zeus.chaos.short:amanda *:* Obviously the output has concatenated my FQDN, but Iwhat I don't know, is if those are all the services that should be listening, and whether those are the correct protocols that they are listening to. seems ok. Thus, I've done rather a lot of research to get this running, and that's not including the many days spent ironing out the kinks in the server issues. This one has me stumped, as the error logs don't even make a peep that something in the system is even running, much less failing... Any help that anyone could provide would be deeply appreciated. After all, as they say, backups are like wives...you don't realize how important they were until you don't have them anymore! Look at the debug files on the client in /tmp/amanda? If there are no files at all, then maybe the directory /tmp/amanda itself is not writeable by the backupuser, e.g. because you once have run some program as root (and then /tmp/amanda gets created as user root, making it unwritable for the real backup user). -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Why amanda creates incremental dumps to holding disk?
Takashi Kurakata wrote: Hi all Amanda-2.5.2p1 on both the server and client. The backup devices are vtapes on the server. I ran amdump on the server. The backup would be full dump (Lv0). After I ran amdump, I found the vtape directory not being correctly set. I Thought that the backup was failure. When I checked the log($logdir/DailySet1/log.mmddhhmmss), I found that the amanda creates incremental dumps. ERROR taper no-tape [No writable valid tape found] WARNING taper changer problem: Virtual-tape directory /var/vtapes does not exist. FINISH planner date 20080123192410 time 1.344 SUCCESS dumper ctamanda /var/test 20080123192410 1 [sec 0.455 kb 10 kps 22.0 orig-kb 10] SUCCESS chunker ctamanda /var/test 20080123192410 1 [sec 0.219 kb 10 kps 191.1] STATS driver estimate ctamanda /var/test 20080123192410 1 [sec 1 nkb 42 ckb 64 kps 10] Would you please answer to my questions follow whether if they are possible or not? 1) Why amanda creates incremental dumps to holding disk? Amanda is supposed to redirect the dumps to a holding disk in the follow case: - No tape drive. - The cable became disconnected. or the wrong tape in the drive or the tape was write protected or ... Why amanda doesn't fail backup and creates incremental dumps to holding disk? When there are tape errors, Amanda falls back to plan B. In that case it tries to proceed by making backup to holding disk, taking into account the reserve parameter in amanda.conf: reserve number Default: 100. The part of holding-disk space that should be reserved for incremental backups if no tape is available, expressed as a percentage of the available holding-disk space (0-100). By default, when there is no tape to write to, degraded mode (incremental) backups will be performed to the holding disk. If full backups should also be allowed in this case, the amount of holding disk space reserved for incrementals should be lowered. Note the default is to reserve 100% for incremental backups. If your holdingdisk is very large, and you want level-0 backups as well, lower the number (even reserve 0 is allowed). 2) Do you judge that which module is it? Do you know whether to judge that which module is Lv1? Planner? Driver? Planner decides the diferent levels for each DLE. But planner also provides a plan B for driver, so that when there suddenly are tape errors, driver can switch some full backups to an incremental when the unreserved holdingdisk space is full. 3) Why amanda-2.5.0p2 isn't creates incremental dumps to holding disk? Neither this time nor the same thing didn't intermited though did by amanda-2.5.0p2 (amanda2.5.0p2-4) that the bundle was done with RHEL5. Why amanda-2.5.0p2 isn't creates incremental dumps to holding disk? I'm not sure I understand the question... -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: selfcheck request timeout
Kermit Short wrote: Thanks, Paul, for your helpful responses! As you requested, I've tried to run the amadmin command but it seems this version doesn't know that parameter of amadmin: [EMAIL PROTECTED]:~$ amadmin DailySet1 config amadmin: unknown command config Oops, my mistake... I meant: /usr/sbin/amadmin xx version However, as this is a binary package, I'm not entirely certain of all the options used to build it. The package notes indicate that the Amanda User is backup with group backup. And verify if backup is indeed what was compiled in the configure options by the command above. Then also make sure that backup is not a duplicate user in the password file! And as general list of things to check, see: http://wiki.zmanda.com/index.php/Selfcheck_request_failed The permissions for the amandahosts file are: -rw-rw 1 backup backup 120 2008-01-21 22:06 amandahosts Better make that 400, instead of 660. There is indeed a /tmp/amanda directory as follows, but it is empty. Permissions, as you see, are set correctly. drwx--S--- 2 backup backup 4096 2008-01-23 00:45 amanda That is a weird problem!!! Try to remove that directory. It will be created automatically next time amandad runs. Try to run the command amandad manually. It should stop after 30 seconds (unless you type some garbage, and then it stops immediatly). Verify if xinetd starts amandad when connected to the port with netcat: nc -u localhost 10080 and verify in another window if the binary amandad runs. If you do not type anything amandad will exit after 30 seconds. Any other messages in the syslog output of xinetd or other places ? Neither root nor backup users on the system are receiving mail from the amanda process, failure notices or otherwise. With no helpful logs or other output, I can't tell where things are failing, so if anyone has any gotchas, I'd love to hear about it! Thanks for the help Paul! -Kermit Paul Bijnens wrote: On 2008-01-23 04:40, Kermit Short wrote: Hey gurus! I've checked many many resources about this error and I've done just about everything they've asked, but I can't get rid of it, and I don't really even know if it's a show stopper or not. Here's my situation: [...] backup set name. In installing/configuring the client, I've done the following: customized the /etc/amandahosts (symlinked by ~backup/.amandahosts) especially here: are the permissions of this file correct? Is that path up top ~backup/.amandahosts secure (not writeable by anone but root or backup-user?) customized the /etc/hosts.allow for the amandad, amindexd, and amidxtaped daemons customized the /etc/services to register the 3 above daemons as services for the xinetd superserver created a drop file in /etc/xinetd.d with the standard recommended parameters and restarted the xinetd superserver OK. we have to believe you that you did not make any error in there. We cannot verify that part of the setup. something that wasn't mentioned on any of the FAQs and mail lists was to change the group on the 3 above daemons to the backup group (the group the 3 above daemons??? which ones? That also seems to imply that you have a discrepancy between the username that is compiled into the executable ( with the configure option --with-user=...) and the one that you use to run the set of programs. They should match! What is the output of amadmin xx config and look at the expected username, in the configure options. specified for the backup user which will be the user running amanda) IPTABLES is not running on this host, so it isn't a firewall issue, unless the port calls go running about the LAN rather than staying internal, which I can't imagine is happening. netstat -a |grep amanda reports the following: tcp0 0 *:amandaidx *:* LISTEN udp0 0 zeus.chaos.short:amanda *:* Obviously the output has concatenated my FQDN, but Iwhat I don't know, is if those are all the services that should be listening, and whether those are the correct protocols that they are listening to. seems ok. Thus, I've done rather a lot of research to get this running, and that's not including the many days spent ironing out the kinks in the server issues. This one has me stumped, as the error logs don't even make a peep that something in the system is even running, much less failing... Any help that anyone could provide would be deeply appreciated. After all, as they say, backups are like wives...you don't realize how important they were until you don't have them anymore! Look at the debug files on the client in /tmp/amanda? If there are no files at all, then maybe the directory /tmp/amanda itself is not writeable by the backupuser, e.g. because you once have run some program as root (and then /tmp/amanda gets created as user root, making it unwritable for the real backup user). -- Paul Bijnens, xplanation
Re: selfcheck request timeout
Kermit Short wrote: Hi Marc, thanks for your ideas! The system does indeed resolve it's full FQDN both through dig/nslookup, but pinging the IP address does not resolve the hostname. Not sure if this is an issue? The output from your netstat command does indeed list the correct ports and services listening from xinetd: [EMAIL PROTECTED]:/tmp$ netstat -taun | egrep :1008. tcp0 0 0.0.0.0:10082 0.0.0.0:* LISTEN tcp0 0 0.0.0.0:10083 0.0.0.0:* LISTEN udp0 0 192.168.34.15:10080 0.0.0.0:* I would expect to see amandad listen on ALL interfaces, especially including localhost!! Verify xinetd configuration for amandad. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: NFS mount tar incremental problem
Jordan Desroches wrote:
Hello all,
I've been having a problem with incremental dumps on a NFS mounted
Netapp. AMANDA runs great until I reboot the client (or remount the NFS
shares on the client). At that point, while calcsize predicts what I
believe is the correct incremental dump size, tar proceeds to do a full
dump of all the NFS mounted files. I believe this has to due with
something changing between mounts that tar is translating as a change to
all files. Upon reading some of the documentation for tar, it indicated
that in the incremental dump gnutar-lists, there should be a 1
preceding every entry to indicate that the file is NFS mounted because
(Quoting
http://www.gnu.org/software/automake/manual/tar/Incremental-Dumps.html;):
Metadata stored in snapshot files include device numbers, which,
obviously is supposed to be a non-volatile value. However, it turns out
that NFS devices have undependable values when an automounter gets in
the picture. This can lead to a great deal of spurious redumping in
incremental dumps, so it is somewhat useless to compare two NFS devices
numbers over time. The solution implemented currently is to considers
all NFS devices as being equal when it comes to comparing directories;
this is fairly gross, but there does not seem to be a better way to go.
Here is an example from one of my gnutar-lists, showing what I believe
are preceding zeroes, indicating that tar thinks that the files are not
on NFS:
[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]/unclaimed_afs/nmlhome/mcbride/.desktop-nauset.dartmouth.edu/[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@./spacescience/web/wl/per/[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@./paulsen/MAC_Keith/Mac_NIH/Proposals/Breast
PPG/Original Proposal '98/Letters^@
Here's how the FS is mounted in /etc/fstab:
192.168.0.2:/vol/research /mnt/thayerfs/research nfs
hard,rsize=32768,wsize=327680 0
And here is an example disk list entry:
tardis /mnt/thayerfs/research_p-z /mnt/thayerfs/research {
nocomp-test
include ./[p-zP-Z]*
}
Has anyone run into this problem, or know how to fix it?
Very related to this:
http://wiki.zmanda.com/index.php/Tar_dumps_every_file_in_a_level-1_backup_after_a_hardware_change
and fixing (each time you have the change!!) it with this script:
http://www.gnu.org/software/tar/utils/tar-snapshot-edit.html
This is actually a gnutar problem...
--
Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, *
* F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... Are you sure? ... YES ... Phew ... I'm out *
***
Re: missing size line from sendbackup and PARTIAL
Sorry, for replying late -- very very busy here... See below On 2008-01-10 16:54, Jean-Francois Malouin wrote: * Paul Bijnens [EMAIL PROTECTED] [20080110 04:40]: On 2008-01-09 22:55, Jean-Francois Malouin wrote: Hi, Amanda-2.5.2p1 on both the server (Debian/Etch) and client (irix-6.5.x). In a amreport this morning I got a DLE with: FAILURE AND STRANGE DUMP SUMMARY: yorick /data/nih/nih1 lev 0 FAILED [missing size line from sendbackup] yorick /data/nih/nih1 lev 0 FAILED [missing size line from sendbackup] The backup program (gnutar for this DLE) normally should have ended with a last line indicating the total size of the backup. However, Amanda did not find that line. But because that does not mean that you want be able to do at least something with the partial backup, Amanda did not discard the whole backup image. (Also because, when e.g. dumping to tape, Amanda will not rewind and overwrite an invalid backup image -- too difficult and maybe some intelligent human can still do something with the partial backup.) But Amanda still marks it FAILED, and will try next time to make a decent backup. [...] Need to find out why it broke off. Look on the client in the sendbackup.DATETIME.debug file Here they come, attached, for the initial run and the retry. What is puzzling me is that the entire image must have first been dumped on the holding disk on the server, then taped. Is it possible that I have hit some timeout. amanda.conf on the server has the following timeouts: ctimeout 360 dtimeout 12960 etimeout 12960 Notice that the first sendbackup on the client failed after 37676s while the retry did only after 5397s... The log files show: sendbackup: time 223.775: started backup sendbackup: time 37676.088: index tee cannot write [Broken pipe] ... sendbackup: time 86.306: started backup sendbackup: time 5397.634: index tee cannot write [Broken pipe] Maybe this is the same problem as described here: http://wiki.zmanda.com/index.php/Mesg_read:_Connection_reset_by_peer ALso have a look in the accompanying runtar.DATETIME.debug files or see if you find any core files from around that time in the same debug directory. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: missing size line from sendbackup and PARTIAL
On 2008-01-09 22:55, Jean-Francois Malouin wrote: Hi, Amanda-2.5.2p1 on both the server (Debian/Etch) and client (irix-6.5.x). In a amreport this morning I got a DLE with: FAILURE AND STRANGE DUMP SUMMARY: yorick /data/nih/nih1 lev 0 FAILED [missing size line from sendbackup] yorick /data/nih/nih1 lev 0 FAILED [missing size line from sendbackup] The backup program (gnutar for this DLE) normally should have ended with a last line indicating the total size of the backup. However, Amanda did not find that line. But because that does not mean that you want be able to do at least something with the partial backup, Amanda did not discard the whole backup image. (Also because, when e.g. dumping to tape, Amanda will not rewind and overwrite an invalid backup image -- too difficult and maybe some intelligent human can still do something with the partial backup.) But Amanda still marks it FAILED, and will try next time to make a decent backup. DUMP SUMMARY: DUMPER STATS TAPER STATS HOSTNAME DISK L ORIG-MB OUT-MB COMP% MMM:SSKB/s MMM:SS KB/s - yorick /data/nih/nih1 0 N/A 84080-- N/A N/A 22:33 63615.7 +PARTIAL Checking what 'adadmin info' tells me for it: amadmin left2 info yorick /data/nih/nih1 Current info for yorick /data/nih/nih1: Stats: dump rates (kps), Full: 21651.2, 23245.9, 22401.7 Incremental: 80.9, 97.4, 157.8 compressed size, Full: -100.0%,-100.0%,-100.0% Incremental: -100.0%,-100.0%,-100.0% Dumps: lev datestmp tape file origK compK secs 0 20080102 av24-2_left2_U00041L3 34 973979040 973979040 44985 1 20080107 av24-2_left2_U00040L3 6 97850 97850 1209 Note the date of the level 0! That is not the date of this morning. The level 0 from this morning is not considered a valid level 0 by Amanda. Seems good but trying to restore the image to disk with amfetchdump fails with: amfetchdump -p left2 yorick /data/nih/nih1 20080108 | tar -xpGf - So you try to restore the one from this morning... ... amfetchdump: 34: restoring split dumpfile: date 20080108 host yorick disk /data/nih/nih1 part 17/17 lev 0 comp N program /usr/freeware/bin/tar amfetchdump: Search of av24-2_left2_U00041L3 complete tar: Unexpected EOF in archive tar: Unexpected EOF in archive And, indeed, the backup image on this tape is a partial backup, that is prematurely broken off. tar: Error is not recoverable: exiting now Any ideas? Need to find out why it broke off. Look on the client in the sendbackup.DATETIME.debug file -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: can't make on Solaris 10: Failed make install!
On 2008-01-09 16:39, Byarlay, Wayne A. wrote: Yes; in fact here's a tall screen grab of the tail end of the make install: [...] ranlib /usr/local/lib/libamanda.a ../libtool: line 6226: ranlib: command not found *** Error code 127 The program ranlib makes an index of the archive to speed up the linking. In modern compilers that is already done by the program while creating the archive. So it does not exist anymore. I remember from a long time ago ( 8 years?) that on some platforms for compiling open source I had to make a command randlib that effectivily did: ar -s or, if that would not work, simply: /bin/true *But* how did you get there? The ./Configure program should have already found out about that. Did you do a make distclean andrun ./Configure again, after you changed your path? If not, you probably are still running with the (cached) configure results from the wrong compiler. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: can't make on Solaris 10: Failed make install!
On 2008-01-09 17:40, Byarlay, Wayne A. wrote: Starting from scratch now, yields the following similar but different all-recursive error. I got rid of my old Amanda-2.5.2p1 directory and did a fresh tar -xvf and then a fresh ./configure. My path is: -bash-3.00# echo $PATH /usr/bin:/usr/local/bin:usr/sfw/bin:/usr/ccs/bin !! Missing slash before usr. /usr/bin:/usr/local/bin:/usr/sfw/bin:/usr/ccs/bin I see two gmakes on my machine: -bash-3.00# ls -hal /usr/sfw/bin/gmake -r-xr-xr-x 1 root bin 150K Jan 22 2005 /usr/sfw/bin/gmake -bash-3.00# ls -hal /opt/sfw/bin/gmake -r-xr-xr-x 1 root bin 150K Jan 5 2005 /opt/sfw/bin/gmake I use the slightly more recent one in sfw, I assume. And although /usr/sfw/bin is in my $PATH, I cannot just gmake. I have to type /usr/sfw/bin/gmake. Is this a problem? Yes, due to the missing slash. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Asking for new tapes when an old tape should be available
On 2007-12-28 18:15, [EMAIL PROTECTED] wrote: I got into a situation where I missed tape changes for a few days, so my backups we put to the holding disk. In the process of dumping the data I inadvertently dumped two days to the same tape when messed up my tape pattern. To fix that I decided to mark the corresponding unneed tape (107) as no-reuse. After that Amanda insists on having a new tape, instead of the next one in line (108). I have reset 107 to be reusable, but amanda wont go back and reuse it now either. Amanda insists on not overwriting the last tapecycle of tapes. You are allowed to have more tapes in the cycle than the tapecycle parameter, so just lower the that parameter in the amanda.conf I have the amanda.conf tapecycle set to 20, but there are 30 real tapes in the cycle. That way amanda always accepts one of the oldest 10 tapes. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: 2.4 vs 2.5
On 2007-12-20 08:13, [EMAIL PROTECTED] wrote: Hi all, Recently I tried to add a host to our backup system. The client was running amanda 2.5, the server 2.4 After a few tries I gave up and installed 2.4 on the client as well (problems with the authentication) These are the new configuration requirements for 2.5.1 and later clients (or servers): http://wiki.zmanda.com/index.php/Configuring_bsd/bsdudp/bsdtcp_authentication I understand that it's best that I also upgrade the server to 2.5 before I do the clients. I have no problem with that, but I do have a few questions: Are there any issues I should be aware of ? Can I re-use the configuration files ? Can the clients be upgraded after the server or should it all be done simultaneously ? Can anyone shed some light on these matters ? As long as you configure a 2.5.1 or later using the info in the webpage above, all versions after 2.4.2 will interoperate. (Of course, to use the newer features, the client or server must be newer too.) -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: amreport analysis
Dustin J. Mitchell wrote:
On Dec 14, 2007 11:11 AM, Tom Herrera [EMAIL PROTECTED] wrote:
? gtar: ./mnt/cdrom: Cannot savedir: Input/output error
? gtar: ./mnt/cdrom: Warning: Cannot savedir: Input/output error
These are a problem. You should add an exclude for /mnt to your DLE:
brain.milw / {
WHATEVER_DUMPTYPE_YOU_HAD
exclude list append ./mnt
} -1 le0
| gtar: ./lib/dev-state/gpmctl: socket ignored
| gtar: ./lib/dev-state/log: socket ignored
| gtar: ./opt/IBMHTTPServer-V1.x/logs/siddport: socket ignored
| gtar: ./opt/IBMHttpServer/conf/socket.26158: socket ignored
| gtar: ./opt/IBMHttpServer/conf/socket.26307: socket ignored
| gtar: ./opt/IBMHttpServer/conf/socket.26386: socket ignored
| gtar: ./opt/IBMHttpServer/conf/socket.27585: socket ignored
| gtar: ./opt/save-IBMHTTPServer/logs/siddport: socket ignored
These are harmless, but you can exclude them, too, if you'd like.
Minor correction: they are harmless indeed, but you do not need
to exclude them, because Amanda expects that output, and classifies
it as normal. You will not see these normal lines in the report
unless there are also error or strange lines.
Each output line of the dumpprogram (gtar above) is classified
as normal All the other lines are flagged with a question mark.
When the return code of gtar indicates success, but contains
unexpected lines, Amanda flags the dump as strange, but still puts
in on tape. Amanda leaves it up to you to interpret the strange
lines, and see if they are harmful or not.
When the return code of gtar indicates a failure, those non-normal
output lines are probably the cause of the error.
When informing you about strange or error in the report, Amanda
always shows the complete output, including the normal lines, to
allow you to interpret those lines in the context.
Re: [Fwd: UB DRD AMANDA VERIFY REPORT FOR DRD021]
Lawrence McMahon wrote: Michael; I should also have included that when I did the amverify, I got the following in /var/adm/messages: Dec 14 13:39:13 apocalypse.acsu.buffalo.edu scsi: [ID 107833 kern.warning] WARNI NG: /[EMAIL PROTECTED],0/[EMAIL PROTECTED]/[EMAIL PROTECTED],1/[EMAIL PROTECTED],0 (st12): Dec 14 13:39:13 apocalypse.acsu.buffalo.edu Error for Command: read Error Level: Fatal Dec 14 13:39:13 apocalypse.acsu.buffalo.edu scsi: [ID 107833 kern.notice] Requested Block: 12495 Error Block: 12495 Dec 14 13:39:13 apocalypse.acsu.buffalo.edu scsi: [ID 107833 kern.notice] Vendor: QUANTUMSerial I'm not sure what you need more to come to the conclusion that you have a broken disk (or a bad scsi cable?).
Re: [Fwd: UB DRD AMANDA VERIFY REPORT FOR DRD021]
Lawrence McMahon wrote: FAILED AND STRANGE DUMP DETAILS: /-- ubcard.car /opt lev 1 STRANGE sendbackup: start [ubcard.card.buffalo.edu:/opt level 1] sendbackup: info BACKUP=/usr/sbin/ufsdump sendbackup: info RECOVER_CMD=/usr/sbin/ufsrestore -f... - sendbackup: info end ? Unable to create temporary directory in any of the directories listed below: ? /tmp/ ? /var/tmp/ ? / ? Please correct this problem and rerun the program. Yes do that. Or your disk is 100% full, or you the disk became readonly due to a filesystem failure, or equally real bad events.
Re: Fw: Intermittent timeouts
The debug you sent is not enough. There should a debug file for each request from the server. The one you sent is the first request, a noop, which the server uses to learn the capabilities of the client. The next request should be a sendsize request packet. Show that one. On 2007-12-12 13:36, Keith Edmunds wrote: Anyone have any suggestions to help move this forward? Thanks! Begin forwarded message: Date: Thu, 6 Dec 2007 15:44:10 + From: Keith Edmunds [EMAIL PROTECTED] To: amanda-users@amanda.org Subject: Intermittent timeouts Hi all We're backing up ten hosts using Amanda; one of them intermittently gets timeouts while all others are OK. The errors look like: [in the log file] FAIL planner bastion /dev/da2s1c 20071205 0 [Request to bastion timed out.] [in the amdump file] planner: time 30.231: error result for host bastion disk /dev/da2s1c: Request to bastion timed out. There are six partitions being backed up on the system in question; if I comment out one or more the disklist entries, the backups run successfully. They've also run successfully in the past on all six partitions (this problem started a couple of months ago). The last part of the debug log on the client server looks like this: amandad: time 0.028: got packet: Amanda 2.4 REQ HANDLE 002-E83A0280 SEQ 1196885402 SECURITY USER amanda SERVICE noop OPTIONS features=feff9ffe0f; amandad: time 0.029: sending ack: Amanda 2.4 ACK HANDLE 002-E83A0280 SEQ 1196885402 amandad: time 0.030: bsd security: remote host fermi user amanda local user operator amandad: time 0.033: amandahosts security check passed amandad: time 0.033: running service noop amandad: time 0.033: sending REP packet: Amanda 2.4 REP HANDLE 002-E83A0280 SEQ 1196885402 OPTIONS features=feff9ffe0f; amandad: time 0.034: got packet: Amanda 2.4 ACK HANDLE 002-E83A0280 SEQ 1196885402 amandad: time 0.034: pid 89078 finish time Wed Dec 5 20:10:00 2007 I've increased etimeout from 1800 to 3600 but there's been no improvement. I'm looking for any ideas to either resolve this problem or to gather more information to find the cause. Thanks, Keith -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Reusing a tape
Marc Muehlfeld wrote:
Yogesh Hasabnis schrieb:
I would like to use the same tape again. What do I need to do to
get this done (ie to use the same tape again)?
Did I understand you right: You backuped your clients and one DLE failed.
And now you want to rerun the complete set on the same tape? In this case
you can relabel your tape
amlabel -f {Set} FULLBACK-10
Amanda ll tread the tape as new. But make sure that your really want to
overwrite the tape. You ll lose the backup of that day.
I usually do: (actually I avoid it, but anyway, if really needed)
First inform amanda that this tape is no longer valid:
amrmtape theconfig FULLBACK-10
Now amanda will clear the indexes etc, but also adjust the bookkeeping
of what levels were successfully backed up. If you omit this, then
amanda will e.g. plan a level 2, based on the level 1 DLE that did not
had errors, but was stored on that tape.
Then label it again, using the -f flag:
amlabel -f theconfig FULLBACK-10
Re: how to recover the whole month backups
On 2007-11-30 12:40, fedora wrote: Hi guys, Normally I do amrecover command to restore backups. Here is my way to recover: amrecover sethost ind.indo.com amrecover setdisk /var/lib/mysql amrecover history for config DailySet1 200- Dump history for config DailySet1 host ind.indo.com disk /var/lib/mysql 201- 2007-11-29-03-00-01 1 DailySet1-26:7 201- 2007-11-28-03-00-01 1 DailySet1-23:18 201- 2007-11-27-03-00-01 1 DailySet1-22:10 201- 2007-11-26-09-45-31 0 DailySet1-20:15 201- 2007-11-23-03-00-01 0 DailySet1-18:12 201- 2007-11-22-03-00-01 0 DailySet1-17:15 201- 2007-11-21-03-00-01 0 DailySet1-16:5 201- 2007-11-20-03-00-01 0 DailySet1-15:5 201- 2007-11-19-03-00-01 0 DailySet1-30:5 201- 2007-11-18-03-00-02 0 DailySet1-03:5 201- 2007-11-17-03-00-01 0 DailySet1-01:6 201- 2007-11-16-03-00-01 0 DailySet1-13:6 201- 2007-11-15-03-00-01 0 DailySet1-12:8 201- 2007-11-14-03-00-02 1 DailySet1-11:2 201- 2007-11-13-03-00-01 1 DailySet1-10:2 201- 2007-11-12-03-00-01 1 DailySet1-09:10 201- 2007-11-11-03-00-02 1 DailySet1-08:2 201- 2007-11-10-03-00-01 1 DailySet1-07:3 201- 2007-11-09-15-50-58 1 DailySet1-06:49 201- 2007-11-08-03-00-02 1 DailySet1-06:36 201- 2007-11-07-03-00-02 1 DailySet1-06:21 201- 2007-11-06-16-46-15 1 DailySet1-06:6 200 Dump history for config DailySet1 host ind.indo.com disk /var/lib/mysql amrecover add * Added dir /Telkomsel_Wallpaper/ at date 2007-11-26-09-45-31 Added dir /Telkomsel_Wallpaper/ at date 2007-11-29-03-00-01 Added dir /Telkomsel_TrueTone/ at date 2007-11-26-09-45-31 Added dir /Telkomsel_TrueTone/ at date 2007-11-29-03-00-01 --- output truncated --- 1) If I do add *, it i'll add the current backup or the whole month? What I have found when I add * it only added backup for 26 and 29 Nov 2007 for the same DB even backup is running daily. You request to restore the files in the state as found on the setdate command (default to today). To accomplish this Amanda looks in its history add collects all necessary levels (level 0, last level 1, last level 2, etc.). 2) I can choose the date if I want to recover only for that day but if I want to recover the whole month how?. I am a bit confused. Actually I want to restore backup for the whole month before tapes being rotated. You're confused about the concepts of backups, or at least, you're using terms nobody else understands. What is your definition of recover a whole month versus recover only for that day? You can recover the state that existed when the backup ran (for a nightly run each weekday that are 5 moments. You have to choose one of them. You cannot take them all 5. You can take a picture on your birthday, and you can look at each picture taken previously and watch growing older. But you cannot look at 1 picture where all your ages are on the same picture (unless you use gimp and copy/paste each face next too the previous one -- just as you can restore each backup in a different directory, and indeed have a whole month restored). -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: planner timeout
On 2007-11-27 18:13, René Kanters wrote: Hi, I have been running into problems that some of my systems are heavily used for long computations making them somewhat less responsive. Last night I ran into the issue that four systems did not send acknowledgments back to the dumper on time during the planning process: planner: ERROR Request to werner.richmond.edu failed: timeout waiting for ACK I looked into allowing more time for that stage, which I believe etimeout should allow, but my amanda.conf has 'etimeout 600' in it while the planner's debug file ends with: security_seterror(handle=0x3038a0, driver=0xa2a0c (BSD) error=timeout waiting for ACK) security_close(handle=0x3038a0, driver=0xa2a0c (BSD)) planner: time 29.898: pid 3734 finish time Tue Nov 27 00:45:36 2007 suggesting that it still only waits for 30 seconds. planner sends a packet to the client(s) and it expects at least an UDP ACK-packet back within 30 seconds, indicating that the client did receive at least the request. The etimeout is the time that planner will wait for the packet with the different size estimates from the client, which will usually take more than 30 seconds. Am I setting the wrong timeout? So it seems you can't even get a reply back. Firewall issues? -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: planner timeout
René Kanters wrote: Hi Paul, thanks for getting back to me on this. I don't think it is firewall issues because up till now the backup ran fine. Actually it ran fine last night. Nobody was doing any big calculations on it, which could have affected that. Also, so maybe something else strange on the network was going on. Is there a way to affect the 30 sec timeout for the planner's ACK timeout? It's hardcoded. Anyway, no reasonable program should be expected to send a simple ACK-packet back taking more than 30 seconds :-) If it can't then anything really bad is wrong, like crashing client programs, firewalls, routing problems etc. Cheers, René On Nov 29, 2007, at 10:24 AM, Paul Bijnens wrote: On 2007-11-27 18:13, René Kanters wrote: Hi, I have been running into problems that some of my systems are heavily used for long computations making them somewhat less responsive. Last night I ran into the issue that four systems did not send acknowledgments back to the dumper on time during the planning process: planner: ERROR Request to werner.richmond.edu failed: timeout waiting for ACK I looked into allowing more time for that stage, which I believe etimeout should allow, but my amanda.conf has 'etimeout 600' in it while the planner's debug file ends with: security_seterror(handle=0x3038a0, driver=0xa2a0c (BSD) error=timeout waiting for ACK) security_close(handle=0x3038a0, driver=0xa2a0c (BSD)) planner: time 29.898: pid 3734 finish time Tue Nov 27 00:45:36 2007 suggesting that it still only waits for 30 seconds. planner sends a packet to the client(s) and it expects at least an UDP ACK-packet back within 30 seconds, indicating that the client did receive at least the request. The etimeout is the time that planner will wait for the packet with the different size estimates from the client, which will usually take more than 30 seconds. Am I setting the wrong timeout? So it seems you can't even get a reply back. Firewall issues? -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: [Mtx-general] mtx error, used with amanda
On 2007-11-26 15:36, Craig Dewick wrote: On Mon, 19 Nov 2007, Brian Cuttler wrote: PS. I'm getting a new cleaning tape for my L9 this week and wondered if anyone has thought more about how to get cleaning tape usage incorporated into the Amanda schema somehow? I have left a spare slot in the array cartridge for the cleaning tape to sit in. The tape config file in the amanda config directory has options for cleaning slot and tape count between cleanings, is that what you are thinking of ? Yes, just wondering if that can be used to automate cleaning tape usage or whether cleaning cycles need to be done manually. I don't know if Amanda has a way to detect when a drive says it needs cleaning. Even humans have trouble to know when a drive needs cleaning :-) The cleaning tapes in DDS drives are lightly abrasive to the magnetic heads, so they should not be used unless you know the drive is dirty and cleaning is the lesser of the two evils. You get hints about cleaning when you get excessive read or write errors on the drive. But besides dirty heads, there are many other causes for read/write errors, like temperature, humidity, bad quality of the tapes (only this one, the whole batch you purchased?). Not all tape drives have cleaning tapes that could damage the head, but, I would not let a Amanda automatically clean the drive whenever there was some error with some tape. Or the logic behind the algorithm should be more clever than me. Anyway, a good tapedrive should not need cleaning too frequently. I never had to clean my LTO-tape drives up to now, 3 years age, no premature write error encountered yet; they just work, as you would expect from decent hardware. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
add option --occurrence to gnutar amrecover command [Was: Re: Problems accessing my new autoloader]
On 2007-11-23 09:35, Francis Galiegue wrote: Le Friday 23 November 2007 09:17:01 Marc Muehlfeld, vous avez écrit : One additional info to my last mail (sorry. I just replied to the sender, and not to the list, thats why I let it below): 5 Minutes after I send this mail, the silent amrecover finished it's work and the files where recovered. But why does it take 45 minutes to restore one 45kb file? On the tape were just 9.1 GB in 60 files, so it should be finished fast. Also its a fast LTO3 drive connected to a fast Adaptec 29160 SCSI card Tar will unroll ALL the archive to find your file. Not sure why it doesn't stop if it finds the file at the beginning/middle of the archive though. Because a file can appear multiple times in a tar archive, and the normal semantics says that it needs the last version, unless specified on the command line. In gnutar you may store files multiple times in an archive. e.g. $ gtar -cf /tmpk/multi.tar file1 file1 file1 $ gtar -tf /tmp/multi.tar and note the file appears 3 times. You can even append to an existing archive: $ date file1 $ gtar -rf /tmp/multi.tar file1 and when you restore, you get last version only. Or be more specific about a specific occurrence: $ gtar --occurrence=3 -tvf /tmp/multi.tar Amanda will *not* store multiple files in a tar archive, and we could add indeed the option --occurrence (the default=1) for extracting with amrecover and then, according to the info page, gnutar will terminate without scanning to the end of the archive. This could speed up the restore process considerably for the few files restores, the most frequent requests. So the default restore command would become: gtar --occurrence --numeric-owner --xpGvf - extractlist Patch against 2.5.2p1: file recover-src/extract_list.c: --- extract_list.c_ORIG 2007-11-23 10:17:22.0 +0100 +++ extract_list.c 2007-11-23 10:18:36.0 +0100 @@ -1773,7 +1773,7 @@ #endif case IS_TAR: case IS_GNUTAR: -extra_params = 4; +extra_params = 5; break; case IS_SAMBA_TAR: extra_params = 3; @@ -1824,6 +1824,7 @@ case IS_TAR: case IS_GNUTAR: restore_args[j++] = stralloc(tar); + restore_args[j++] = stralloc(--occurrence); /* do not scan the rest of the archive if file found once */ restore_args[j++] = stralloc(--numeric-owner); restore_args[j++] = stralloc(-xpGvf); restore_args[j++] = stralloc(-); /* data on stdin */ -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: Problems accessing my new autoloader
Marc Muehlfeld wrote: Francis Galiegue schrieb: And what if you try to read the label by hand? Ie, try at the command line: dd if=/dev/nst0 bs=32k count=1 Does anything appear or do you have an error? # dd if=/dev/nst0 bs=32k count=1 AEAFDA: TAPESTART DATE P TAPE KAS001 AEAFDA ?? that should have been AMANDA. What is changing your M's into E's and N's into F's ?? 014 1+0 Datensätze ein 1+0 Datensätze aus 32768 Bytes (33 kB) kopiert, 0,0182859 s, 1,8 MB/s
Re: could not connect DATA stream
sorry for top-posting, but I give up. Or, my english is not good enough, or you succeed in answering clear questions with misty and foggy answers. On 2007-11-19 04:34, fedora wrote: Relabeling erases the tape, indeed. But why relabel it? Do you mean that you keep inserting the same tape over and over again until that one client succeeds? keep inserting for each dump to different tapes I don't understand your answer. Why not first trouble that one client? Make a diskist file with one small DLE from that client. And while testing the firewall-issues with that client, you do not have to insert a tape; just dump to holdingdisk. How to dump to holdingdisk? By reading the manual. (e.g set reserve 0 and leave out the tape.) The reason why the images are not on the tape is worded in plain english above. Was there a tape in? Yes got tapes. Ok, you got them, but were they in the drive ? :-) Was it a tape that could be reused (older than tapecycle?) Yes. it is reused as in tapelist. Was it write protected? Yes. That's weird, because if it is write protected, than you cannot write to it. What did amcheck say about that tape before? amcheck said no problem found That's weird too, because amcheck would complain that the tape is write protected,, at least if you try the amcheck -w option (that you probably should do!). How did you allow those? Did it suceeed? Let's see the commands, so that we can verify them. Does your firewall do port-NAT interfering with the port ranges you specified? on client's shorewall firewall: ACCEPT net:x.x.x.x $FW udp 10080 ACCEPT net:x.x.x.x $FW tcp - 5:50100 whereby x.x.x.x is server I dont have firewall on server but in front of server got pfsense firewall. So in pfsense: UDP * * 10.200.30.150 10080 * TCP * * 10.200.30.1505 - 50100 * whereby 10.200.30.150 is amanda server (internal ip). It is mapping 1:1 from external ip to internal ip. But, as said above, much easier is to use bsdtcp which is much more firewall friendly. I think something wrong with my pfsense (not to say my rules doesnt work because I can backup for other 10 servers). Weird, it could be connection problem between my problem client and pfsense/sever. I had tried to route my server to use another server as a gateway instead of using pfsense. I can backup successfully. I will try on bsdtcp if that is alternative solutions. Thanks in advanced. ok, try that first. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: can't repeat Mesg read: Connection reset by peer
Takashi Kurakata wrote:
Hi all
I'm sorry if you understand my English.I have the pleasure of writing to
you again.
I examine why this log Mesg read: Connection reset by peer is output.
To want to repeat it, I executed the following commands on the server.
- amdump daily
- iptables -A INPUT -p tcp -d IPaddress of the client -j REJECT
--reject-with tcp-reset
- iptables -A OUTPUT -p tcp -d IPaddress of the client -j REJECT
--reject-with tcp-reset
Ok, so those entries effectivily block any traffic from/to the client,
and immediately send the tcp-reset on each packet.
I executed the command iptables while executing the command amdump.
Unfortunately, I can't repeat it.
The following error messages are output.
- data read: Connection reset by peer
So the first packet that the server sends seems to be a packet on the
data channel. And that gets the tcp-reset back. That results in the
connection reset by peer.
So what exactly are you trying to simulate? You want the tcp-reset only
on the message channel and not on the data channel?
In that case you need to let the data channel open, but firewall only
the message channel instead.
That is difficult because you'll need to find out which of the three
channels that were negotiated is the message channel.
You may trace the UDP traffic on port 10080 for that and dump the
packets. One of those packets has a payload like:
CONNECT DATA 40121 MESG 40122 INDEX 40123
OPTIONS features=feff9ffeff7f;
And there you find the different tcp ports used by the three channels.
And then block only the MESG channel (40122 in the above example).
I checked the URL as follow, but I don't solve the problem.
http://wiki.zmanda.com/index.php/Mesg_read:_Connection_reset_by_peer
I don't understand which problem you're trying to solve.
You cannot solve the problem that you created using the iptables
blocking, with the solution in the webpage that just tries to send
MORE packets to avoid a statefull firewall timing out idle connections.
Those keep alive packets will be the iptables rule as well and
get a tcp-reset as well.
If you have done, could you tell me how to repeat it?
I consider that I can repeat it if the local valiable n returns -1.
common-src/bsd-security.c
do {
n = read(bs-fd, bs-databuf, sizeof(bs-databuf));
} while ((n 0) ((errno == EINTR) || (errno == EAGAIN)
Indeed, when the read syscall returns -1, it failed and in that
case the reason of failure is found in the errno variable.
Do you mean reproduce the error or repeat the error (subtle
difference in meaning).
system configuration as follow:
- server/client
- OS:RHEL5
- amanda:2.5.0p2-4(bundle in RHEL5)
The result was the same though I had tried amanda2.5.2-p2.
Yours sincerely,
Takashi Kurakata
2007/11/15, Takashi Kurakata [EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]:
Hi,all
I do the following with the server to let error message Mesg read:
Connection reset by peer reappear.
- amdump DailySet1
- iptables -A INPUT -p tcp -d IPaddress of the client -j REJECT
--reject-with tcp-reset(during amdump)
- iptables -A OUTPUT -p tcp -d IPaddress of the client -j REJECT
--reject-with tcp-reset(during amdump)
However, the error message is not reproduced.
The following error message data read: Connection reset by peer is
output.
I checked the URL as follow, but I don't solve the problem.
http://wiki.zmanda.com/index.php/Mesg_read:_Connection_reset_by_peer
http://wiki.zmanda.com/index.php/Mesg_read:_Connection_reset_by_peer
Have you reproduced this? If there is some good method, Would you
tell me how to reproduce the error message Mesg read: Connection
reset by peer?
The system constitution is as follow:
- I use amanda2.5.0p2-4(with RHEL5) and amanda2.5.2p1(Source).
- There isn't a firewall between the server and client.
Your prompt reply would be greatly appreciated.
Thanks in advance.
--
Paul Bijnens, XplanationTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ...*
* ... Are you sure? ... YES ... Phew ... I'm out *
***
