Re: flush full dumps only
I have attached a very simple but effective script that I set up to separate full and incremental holding disk files, so that full dumps can be flushed to tape while incrementals remain on disk. The script is almost so simple as to be a joke, but I don't have any idea what kind of bugs can be found. WARNING: The script is provided as-is with no warranty express or implied. If you screw up your amanda backups do not come crying to me. To really make the script work, you will need to change the values of $r and $i for your configuration, run it to see what it will do, and then remove the echo from the beginning of the two lines. If you can't figure out what that means, then you probably shouldn't be screwing around with this stuff anyway and should forget about it. For restores, I think I can flush the latest batch of full dumps, then use a modified amanda.conf that looks in the directory where the incrementals are stored, but I am not sure how amanda decides where holding disk files are located. I might have a use another script to move the incrementals back to the normal holding disk tree, but I will cross that bridge when I come to it, to use a rather hackneyed expression. -- Jonathan Dill [EMAIL PROTECTED] #!/bin/tcsh -f set r=(/snap/amanda-hd) set i=(/snap/amanda-incr) cd $r set dl=(`find . -depth -type d -print`) cd $i foreach d ($dl) if (! -d $d) then echo mkdir -p $d endif end cd $r foreach d ($dl) set fl=(`find . -type f \! -name *.0 -print`) foreach f ($fl) echo mv $f $i/$f end end
Re: flush full dumps only
Hi John,
This simple patch should do a better job, but it should be configurable
by an option in the configuration file:
eg. taping_level [full|incr|all]
Jean-Louis
On Fri, Mar 19, 2004 at 12:00:16PM -0500, Jonathan Dill wrote:
I have attached a very simple but effective script that I set up to
separate full and incremental holding disk files, so that full dumps can
be flushed to tape while incrementals remain on disk. The script is
almost so simple as to be a joke, but I don't have any idea what kind of
bugs can be found.
WARNING: The script is provided as-is with no warranty express or
implied. If you screw up your amanda backups do not come crying to me.
To really make the script work, you will need to change the values of $r
and $i for your configuration, run it to see what it will do, and then
remove the echo from the beginning of the two lines. If you can't
figure out what that means, then you probably shouldn't be screwing
around with this stuff anyway and should forget about it.
For restores, I think I can flush the latest batch of full dumps, then
use a modified amanda.conf that looks in the directory where the
incrementals are stored, but I am not sure how amanda decides where
holding disk files are located. I might have a use another script to
move the incrementals back to the normal holding disk tree, but I will
cross that bridge when I come to it, to use a rather hackneyed
expression.
--
Jonathan Dill [EMAIL PROTECTED]
#!/bin/tcsh -f
set r=(/snap/amanda-hd)
set i=(/snap/amanda-incr)
cd $r
set dl=(`find . -depth -type d -print`)
cd $i
foreach d ($dl)
if (! -d $d) then
echo mkdir -p $d
endif
end
cd $r
foreach d ($dl)
set fl=(`find . -type f \! -name *.0 -print`)
foreach f ($fl)
echo mv $f $i/$f
end
end
--
Jean-Louis Martineau email: [EMAIL PROTECTED]
Département IRO, Université de Montréal
C.P. 6128, Succ. CENTRE-VILLETel: (514) 343-6111 ext. 3529
Montréal, Canada, H3C 3J7Fax: (514) 343-5834
diff -u -r --show-c-function --exclude-from=amanda.diff
amanda-2.4.5b1.orig/server-src/amflush.c amanda-2.4.5b1.new/server-src/amflush.c
--- amanda-2.4.5b1.orig/server-src/amflush.c2003-06-05 13:06:20.0 -0400
+++ amanda-2.4.5b1.new/server-src/amflush.c 2004-03-19 12:40:17.0 -0500
@@ -276,6 +276,7 @@ char **main_argv;
dp = lookup_disk(file.name, file.disk);
if (dp-todo == 0) continue;
+ if (file.dumplevel != 0) continue;
fprintf(stderr,
FLUSH %s %s %s %d %s\n,
diff -u -r --show-c-function --exclude-from=amanda.diff
amanda-2.4.5b1.orig/server-src/driver.c amanda-2.4.5b1.new/server-src/driver.c
--- amanda-2.4.5b1.orig/server-src/driver.c 2004-02-13 09:03:36.0 -0500
+++ amanda-2.4.5b1.new/server-src/driver.c 2004-03-19 12:40:32.0 -0500
@@ -1126,7 +1126,8 @@ void handle_dumper_result(fd)
dp-host-hostname, dp-name);
fflush(stdout);
- enqueue_disk(tapeq, dp);
+ if(sched(dp)-level == 0)
+ enqueue_disk(tapeq, dp);
dp = NULL;
startaflush();
diff -u -r --show-c-function --exclude-from=amanda.diff
amanda-2.4.5b1.orig/server-src/planner.c amanda-2.4.5b1.new/server-src/planner.c
--- amanda-2.4.5b1.orig/server-src/planner.c2004-02-13 09:01:08.0 -0500
+++ amanda-2.4.5b1.new/server-src/planner.c 2004-03-19 12:43:14.0 -0500
@@ -536,7 +536,9 @@ char **argv;
for(holding_file=holding_list-first; holding_file != NULL;
holding_file = holding_file-next) {
get_dumpfile(holding_file-name, file);
-
+
+ if (file.dumplevel != 0) continue;
+
fprintf(stderr,
FLUSH %s %s %s %d %s\n,
file.name,
Re: flush full dumps only
Jonathan Dill wrote: almost so simple as to be a joke, but I don't have any idea what kind of bugs can be found. See below :-) #!/bin/tcsh -f set r=(/snap/amanda-hd) set i=(/snap/amanda-incr) cd $r set dl=(`find . -depth -type d -print`) cd $i foreach d ($dl) if (! -d $d) then echo mkdir -p $d endif end cd $r foreach d ($dl) set fl=(`find . -type f \! -name *.0 -print`) foreach f ($fl) echo mv $f $i/$f end end The last find command is intended to get all but the level zero dumps, but it is not completely correct: Full dumps that were split into chunks have a dot + chunknumber appended after the level .0. I can't come up with a perfect solution, but the find below should work fine if the chunks are not too small (if each filesystem is 100 chunks at most). find . -type f \! -name *.0 \ -a \! -name *.0.[0-9] \ -a \! -name *.0.[0-9][0-9] \ -print at least if we don't have a DLE that ends in .0, and where the incremental backup would end in .0.1. :-( ps. You could replace the first find, mkdir with: cd $r; find . -type d | cpio -pd $i pps. If you don't mind flush the incrementals to tape too, the script could be: cd $r find . -type f \! -name *.0 \ -a \! -name *.0.[0-9] \ -a \! -name *.0.[0-9][0-9] \ -print | tee /tmp/amanda/flushonly | cpio -pdml $i rm `cat /tmp/amanda/flushonly` and you should avoid DLE's with confusing names... -- Paul Bijnens, XplanationTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ...* * ... Are you sure? ... YES ... Phew ... I'm out * ***
flush full dumps only?
Has anyone tried the approach of only flushing full dumps and leaving incremental dumps on disk? I think that this would have roughly the same effect as doing full dumps out of cycle, but I have not had luck so far with the out-of-cycle approach. I think that it should be fairly easy to script, create a parallel directory tree and temporarily move out the incremental dumps that you don't want to flush, for example. The idea would be to leave all of the incremental dumps available on a huge disk, but keep all of the full dumps on tape, so at most one tape should be required for any restore. Since the full dumps take up the most space, clearing them out should leave more space for a longer history of incrementals. At least that is the idea. --jonathan
