Muchael, > 1) if I have multiple DLE's in my disklist, then tell amdump to perform a > level 0 dump of the complete config, each DLE gets written to tape as a > separate dump/tar file (possibly in parts if the tar is > part-size). Is > that right?
Yes > 2) If multiple DLE's are processed in a single level 0 amdump run, with > each DLE << tape-size, then as many as can fit will be written to a single > tape, or possibly spanning tapes. But in any case it won't be a single DLE > per tape. Is that right? That looks like what I've observed so far. Yes, Amanda try to fit as many dle/tape in order to fill-in the tape. > 3) I had figured that when restoring, amrestore has to read in a complete > dump/tar file before it can extract even a single file. So if I have a > single DLE that's ~2TB that fits (with multiple parts) on a single tape, > then to restore a single file, amrestore has to read the whole tape. > HOWEVER, I'm now testing restoring a single file from a large 2.1TB DLE, > and the file has been restored, but the amrecover operation is still > running, for quite some time after restoring the file. Why might this be > happening? Your touching the essence or tapes here: they are sequential access. So in order to access one specifi DLE on the tape, the tape has to position at the very begining of the tape and read everything until it reaches that dle (the nth file on the tape). Then it has to read sequentially all that file containing the backup of a dle to find the file(s) you want to restore. I am not sure about dump, but I am pretty sure that if your tar backup was a file on a disk instead of a file on a tape, it would read sequentially from the begining of the tar file, in a similar way. Then it has to read until the end of the tar (not sure about dump) to make sure that there is no other file(s) satisfying your extraction criteria. So yes, if the file you want to extract is at the begining of your tar, it will continue reading for a certain amount of time after the file has been extracted. > The recover log shows this on the client doing the recovery: > > [root@cfile amRecoverTest_Feb_27]# tail -f > /var/log/amanda/client/jet1/amrecover.20140227135820.debug > Thu Feb 27 17:23:12 2014: thd-0x25f1590: amrecover: stream_read_callback: > data is still flowing > > 3a) Where is the recovered dump file written to by amrecover? I can't see > space being used for it on either server or client. Is it streaming and > untar'ing in memory, only writing the desired files to disk? In the directory from where you started the amrecover command. With tar, it will create the same exact hierarchy, reflecting the original DLE. try: find . -name myfilename -print > 4) To restore from a single DLE's dump/tar file that's smaller than tape > size, and exists on a tape with multiple other smaller DLE dump/tar files, > amrestore can seek to the particular DLE's dump/tar file and only has to > read that one file. Is that right? As mentionned above, seek on a tape is a sequential read of the tape (unless your tape is already positionned on the file x (known) and you want to read from file y, it will need to read only y-x). > So assuming all the above is true, it'd be great if amdump could > automatically break large DLE's into small DLE's to end up with smaller > dump files and faster restore of individual files. Maybe it would happen > only for level 0 dumps, so that incremental dumps would still use the same > sub-DLE's used by the most recent level 0 dump. Yes, but then what happens for level above 0? You have to make your planning by hand and break your dle yourself. > The issue I have is that with 30TB of data, there'd be lots of manual Depending on the size of your tapes, even with many small dle, you will most probably end-up reading the tape from the begining for every restore. If your dle is splitted on many tapes, you will have to read every tape even if the file you want was found on the first tape (I am not 100% sure about that though). > fragmenting of data directories to get more easily-restorable DLE's sizes > of say, 500GB each. Some top-level dirs in my main data drive have 3-6TB > each, while many others have only 100GB or so. Manually breaking these into > smaller DLE's once is fine, but since data gets regularly moved, added and > deleted, things would quickly change and upset my smaller DLE's. > > Any thoughts on how I can approach this? If amanda can't do it, I thought I > might try a script to create DLE's of a desired size based on disk-usage, > then run the script everytime I wanted to do a new level 0 dump. That of > course would mean telling amanda when I wanted to do level 0's, rather than > amanda controlling it. While the script may be a good idea, running it for each level 0 will completely mess up with Amanda: remember, you don't manage/know when Amanda will do a level 0. So you don't know when to run your script. And if you remove a dle from your disklist (because you have splitted it into 2 for example), I am pretty sure that you cannot restore from it anymore (you could with amrestore, but it is though). (and of course if you keep both the old dle and the 2 splitted ones, you will have double back-up). Bests, Olivier > Thanks for reading this long post! > > -M > > [2:text/html Show] > > > -- >
