After struggling around about a week with amanda to get it working in a tapless network backup configuration it finally seems to be doing as it should.
I've put together a little How-To to help others (as well as myself) with the setup of amanda for tapeless operation. It got a bit lengthy I fear but I think it is rather comprehensive. I am posting it here, at the amanda-users mailing list, hoping that it will be archived for further reference. Please feel free to mail me about errors, mishaps, spelling or grammar mistakes as well as suggestions for extension or corrections if you want. I have written this by myself, with help of the following How-To in the Amanda Faq-o-matic archive (borrowed a bit from the structure and a little from contents): http://amanda.sourceforge.net/fom-serve/cache/191.html So here it is: Amanda tapeless network backup operation Institut fuer Nachrichtentechnik und Hochfrequenztechnik Technische Universitaet Wien (Vienna University of Technology) Vienna, Austria Author: Andreas Ntaflos ant at overclockers in the dot at domain Date: 2003-07-30 Contents ======== 1 Assumptions/Requirements 2 Installation 3 Backup configuration 4 Restoring 5 Problems This HOWTO covers amanda 2.4.4p1. - - - - - - - - - - - - - - - - - 1 Assumptions/Requirements ========================== It's helpful/mandatory to .) know what amanda is and what it is used for, .) know UNIX/Linux/shit, .) know where to find config files, .) have a thought-through backup strategy prepared, .) etc... The amanda backup configuration is called 'normal'. The backup server (which generates and stores the backups) is called 'backup'. The backup client (which holds the important data to be backed up) is called 'mobilkom'. There are no tapes. Instead, 'backup' uses a 600GB raid-array of IDE disks. 'backup' and 'mobilkom' are connected via a 100Mbit CAT5 ethernet cable. Wow. 'mobilkom' is accessible from the outside, 'backup' only via 'mobilkom' (mobilkom has two ethernet NICs). Configuration files for amanda are on 'backup', located under /usr/local/etc/amanda/normal. The files are called amanda.conf, disklist, and changer.conf. The backup strategy (subject to change, I believe?) --------------------------------------------------- 7 day dumpcycle, rotating 7 tapes, one for each day. Backups start at Sunday night with a level 0 dump of the data. Each day the backup level gets incre- mented by one. 'mobilkom' has a 400GB raid5 array on device /dev/md0, mounted on /d which holds the data to be backed up. 'backup's 600GB raid array (the 'tapes') is also /dev/md0 (but on 'backup') and mounted on /dump, where a subdirectory for each tape exists, according to the configuration ('normal' in this case) 'backup' also has a mount point /d which is an NFS export share from mobilkom of the data to be backed up (used for restoring, see 'Configuration' and 'Problems'). 2 Installation ============== The installation of amanda involves both client and server. The INSTALL and docs/INSTALL files coming with the amanda distribution describes in detail how amanda can be configured at build-time and what options it provides. Server installation ------------------- Here, the following options have been set: --with-user=amanda --with-group=disk --with-config=normal According to the INSTALL files, amanda can be built and installed thus (as root): # ./configure [OPTIONS] # make # make install This will add a new user amanda to the system, with it's home directory in /var/lib/amanda, which also serves as the state directory. There all the log files will be put as well as the indexing database for use with amrecover. It varies from system to system where the binaries are to installed to, but common places for amanda are: /usr/bin /usr/sbin /usr/local/bin /usr/local/sbin /usr/local/libexec The binaries for the server are the following (possibly incomplete list): amadmin amcheck amcheckdb amcleanup amdd amdump amflush amgetconf amlabel ammt amoverview amplot amrecover amreport amrestore amrmtape amstatus amtape amtapetype amtoc amverify amverifyrun Server configuration -------------------- 1) The amanda services have to be put in the /etc/services file: amanda 10080/udp amandaidx 10082/tcp amidxtape 10083/tcp 2) The indexing capabilities of amanda are also needed, so the services have to be started with inetd. The underlying systems use xinetd, so the config- uration will look as follows: /etc/xinetd.d/amandaidx: service amandaidx { socket_type = stream protocol = tcp wait = no user = amanda group = disk server = /usr/local/libexec/amindexd disable = no } /etc/xinted.d/amidxtape: service amidxtape { socket_type = stream protocol = tcp wait = no user = amanda group = disk server = /usr/local/libexec/amidxtaped disable = no } 3) In order for the client to access the amanda services, they have to have the permissions to do so; they are controlled by the .amandahosts file in the home directory of amanda (e.g. /var/lib/amanda): .amandahosts: localhost amanda localhost.localdomain amanda mobilkom root backup root 4) A cronjob is needed for amanda to execute the backup procedures on a regular basis. amdump is used for the dumping of files and filesystems. It only takes the configuration name as an argument. The cronjob must be done by user amanda. As amanda the following command will create a new cronjob, running every day of the week at 23:00: $ crontab -e SHELL=/bin/sh 00 23 * * * /usr/local/sbin/amdump normal Client installation ------------------- The extracted distribution of amanda contains a directory called client-src. Cd'ing there and issuing make and make install should take care of compiling and installing the necessary client binaries. The default paths are /usr/local/bin and /usr/local/libexec. Client configuration -------------------- 1) The client needs to know about the services running, too. The following entries are to be added to /etc/services: amanda 10080/udp amandaidx 10082/tcp amidxtape 10083/tcp 2) On the client the only service that needs to run is the amandad daemon. It is located unter /usr/local/libexec/amandad. Starting it via inetd: /etc/xinetd.d/amanda: service amanda { socket_type = dgram protocol = udp user = amanda group = disk wait = yes server = /usr/local/libexec/amandad disable = no } 3) Authentication on the client is also needed, so the backup server can connect to it. This is done via the .amandahosts file in ~amanda, too. .amandahosts: backup mobilkom 3 Backup configuration ====================== The three configuration files for the correct backup operation are the aforementioned 'amanda.conf', 'disklist' and 'changer.conf'. They reside on the backup server, under /usr/local/etc/amanda/normal in case of the present configuration of 'backup' and 'mobilkom'. The following describes what lines are to be edited or added to either configuration file. amanda.conf ----------- org "normal" # your organization name for reports mailto "amanda" # space separated list of operators at your site dumpuser "amanda" # the user to run dumps under netusage 600 kbps # maximum net bandwidth for Amanda, in KB per sec dumpcycle 7 days # the number of days in the normal dump cycle runspercycle 7 # the number of amdump runs in dumpcycle days tapecycle 7 # the number of tapes in rotation # This enables a backup cycle of one week, backups on every day, with one tape # per day. runtapes 1 # number of tapes to be used in a single run of amdump tpchanger "chg-multi" # the tape-changer glue script tapedev "file:/dump/tape" # the no-rewind tape device to be used # see later in this document tapetype HARD-DISK # what kind of tape it is; this is defined later in # the config file labelstr "^normal[0-9][0-9]*$" # label constraint regex: all tapes must match # This is the pattern which every tapelabel must match; in case of backup and # mobilkom this would be normal01, normal02, ..., normal99. infofile "/var/lib/amanda/normal/curinfo" # database filename logdir "/var/lib/amanda/normal" # log directory indexdir "/var/lib/amanda/normal/index" # index directory tapelist "/var/lib/amanda/normal/tapelist" # list of used tapes define tapetype HARD-DISK { comment "Hard disk instead of tape" length 601 gbytes } # This defines a tapetype HARD-DISK with a 'length' of 601GB, the size of the # backup array. define dumptype hard-disk-dump { global comment "Back up to hard disk instead of tape using dump" holdingdisk no index yes priority high kencrypt no } define dumptype hard-disk-tar { comment "Back up to hard disk instead of tape - using tar" kencrypt no program "GNUTAR" priority high index yes } # Defining a real dump of the filesystem and tar-ing the contents of the # directory in question. tar runs faster and more platform and more system # indepentent (when using the GNU version of tar, available for virtually # every existing UNIX operating system) than dump. define interface local { comment "a local disk" use 1000 kbps } define interface eth0 { comment "10 Mbps ethernet" use 600 kbps } # Network interface definitions. changer.conf ------------ multieject 0 gravity 0 needeject 0 ejectdelay 0 statefile /var/lib/amanda/normal/changer-status firstslot 1 lastslot 7 slot 1 file:/dump/normal01/ slot 2 file:/dump/normal02/ slot 3 file:/dump/normal03/ slot 4 file:/dump/normal04/ slot 5 file:/dump/normal05/ slot 6 file:/dump/normal06/ slot 7 file:/dump/normal07/ This tells amanda that a tape changer is used; it has 7 slots and each slot bears a tape labeled normal01 through normal07. Full dumps are thus supposed to go on normal01. disklist -------- mobilkom md0 hard-disk-tar The disklist defines which host is backed up, which disk on the host is used and what kind of backup should be made. Tape setup ---------- For this to work the directories /dump/normal* have to be created (by the amanda user) and the 'tapes' have to be labelled accordingly. Creating the 'tapes' on the disk: # mkdir -p /dump/normal01 # mkdir -p /dump/normal02 . . # mkdir -p /dump/normal07 Labelling the tapes: # amlabel normal normal01 slot 1 # amlabel normal normal02 slot 2 . . # amlabel normal normal07 slot 7 For restoring, the current tape is always referred to by /dump/tape; a symbolic link to the currently loaded tape (e.g. normal02) ensures this. /dump/tape has been specified in amanda.conf above. # ln -s /dump/normal02 /dump/tape This should be changed whenever a restore program like amrecover requires another or the next tape to be put into the drive. The tapes are always referred to by label. 4 Restoring =========== The way to restore data backed up on the 'tapes' depends on the dump-type specified in disklist. For 'dump', the UNIX program restore is used in conjunction with amrestore and for 'tar' the GNU program tar (surprised?). The amanda command amrecover can be used on the client to interactively retrieve files indexed by the amanda indexing service during backups. It does not work on the herein discussed setup. See Problems below. On backup and mobilkom, the restoring is done from host backup, but using the NFS export share /d from mobilkom. Thus all extracted and restored files are automatically put on mobilkoms /d partition (the raid5 array). This is important and necessary because when working with hundreds of GB there is no space left for much overhead such as trying to extract data on the backup server first and only afterwards copying it to the backup client. restore ------- The simplest way to work with restore is to pipe the output of amrestore to it and run it interactively from backup (using tape normal02 in this example): # cd /d # amrestore -p -f 1 file:/dump/normal01 mobilkom md0 | restore -ivb 2 -f - amrestore: -p pipes the output, -f 1 'rewinds' the 'tape', mobilkom specifies the host and md0 the disk from which the backups were done. restore: -i provides an interactive shell (much like FTP), -v is verbose, -b 2 sets the blocksize to 2 to avoid error messages concerning 'short reads' and -f - specifies the file to be worked on, in this case coming from the output of amrestore. See the man page for both commands for further options. tar --- Using tar works in a similar way, on backup, too: # cd /d To list the contents of the tar backup file/tape: # amrestore -p -f 1 file:/dump/normal02 mobilkom md0 | tar -tvf - To extract all files on the tape: # amrestore -p -f 1 file:/dump/normal02 mobilkom md0 | tar -xvf - To extract specific files on the tape: # amrestore -p -f 1 file:/dump/normal02 mobilkom md0 | tar -xvf - ./path/to/file/on/archive/here 5 Problems ========== Besides its nice and comprehenisve functionality amanda also seems to have some more or less severe bugs which may or may not happen to be caused by the specific setup in our case (using disk files instead of real tapes and date drives). Two major problems were encountered during testing and configuring amanda. 1) amrecover borking --------------------- 1.1) Stalling TCP connection on extraction - - - - - - - - - - - - - - - - - - - - - Using amrecover on the backup client should work much like an FTP or an inter- active restore session. It is used thus: Change the directory to the disk which used to hold the original files and which is the place where the are to be restored to. # cd /d invoke amrecover, specifying the tape server, index server and the config. # amrecover -C normal -t backup -s backup Here files can be added to an extraction list and then extracted using (you guessed it) extract. At least that's how it should be. AMRECOVER Version 2.4.4p1. Contacting server on backup ... 220 backup AMANDA index server (2.4.4p1) ready. 200 Access OK Setting restore date to today (2003-07-23) 200 Working date set to 2003-07-23. 200 Config set to normal. 200 Dump host set to mobilkom. Trying disk /d ... Trying disk md0 ... $CWD '/d' is on disk 'md0' mounted at '/d'. 200 Disk set to md0. /d amrecover> ls 2003-07-3 testdir1/ amrecover> add testdir1 Added dir /testdir1 at date 2003-07-2 amrecover> extract Extracting files using tape drive file:/dump/normal02 on host backup. The following tapes are needed: normal02 Restoring files into directory /d Continue [?/Y/n]? y Extracting files using tape drive file:/dump/normal02 on host backup. Load tape normal02 now Continue [?/Y/n/s/t]? y The above transcript illustrates how amrecover is operated up to the point where the actual files should be extracted. After issuing the last 'y', the connection to the backup server stalls and sits there more or less idley for hours. No files get extracted or transferred. Solution: None as of the time of writing. Workaround: Using amrestore and an NFS export of the client disk (see above Installation and Configuration). 1.2) Index on index server (backup) outdated, displaying old files - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - instead of newly backupped ones - - - - - - - - - - - - - - - - When using amrecover the program connects to the backup server and queries the index catalogue it made during the backup. Every single file on the backup tapes should thus be accessible. This does not seem to work always. Often the amanda server provides only an outdated (several dump cycles before made) index of the backed up files. It is not possible to recover files from a dump after the outdated index was created. This could be the result of a misconfiguration on the server side or a problem during the extensive tests of amandas functionality on backup. However, since amrecover wouldn't allow for any file recovery as described above, this problem/bug wasn't cared about much. Solution: None as of the time of writing. Workaround: Using amrestore and an NFS export of the client disk (see above: Installation and Configuration). 2) amdump failing to dump to 'tapes', making dumpfiles/tapes on disk - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - inaccessible/corrupting them - - - - - - - - - - - - - - This problem is indicated by the amdump mail report. A line like this shows up: NOTES: driver: mobilkom md0 3 [dump to tape failed, will try again] driver: mobilkom md0 3 [dump to tape failed, will try again] taper: tape normal02 kb 17184 fm 3 [OK] No further information about the failed dump can be found in any of the log files so this is quite a cryptic problem. When trying to re- store from such a 'tape' a message like the following comes up and the restore process borks: [EMAIL PROTECTED] root]# amrestore -f 1 -p file:/dump/normal02 mobilkom md0 | restore -ivb 2 -f - amrestore: 1: reached end of information Verify tape and initialize maps Input is from file/pipe restore: Tape read error on first record Solution: None as of the time of writing. Workaround: Using GNUTAR and dump-type hard-disk-tar uses tar to create an archive of the filesystem or partition in question and stores it on the tape. This works fine and extraction and restoring works too. See above (Restoring). -- Andreas "ant" Ntaflos | "A cynic is a man who knows the price of [EMAIL PROTECTED] | everything, and the value of nothing." Vienna, AUSTRIA | Oscar Wilde
