Hi,
Timothy J. Massey wrote on 29.01.2007 at 01:46:34 [[BackupPC-users] Submitting
an archive via BackupPC_serverMesg in a scriptable way]:
>
> Here are some notes on how to submit an archive job to BackupPC in the
> same way that the GUI does.
thank you for sharing the result of your research. I've hacked together a
Perl script which accepts a few command line options to do what you've
described, copying a lot from what you wrote. I'm not claiming that it's
brilliantly coded, complete, or even works (well, it's meant to ...). I
wanted to try out using BackupPC::Lib functions, and might as well share
the result, in case it's useful to someone.
> [Generation of request file name]
> No matter: the file name is not very important at all: from what I can
> tell, the whole file name could be completely random. So, onward we go.
I'll stick to it in my example for consistency.
> Here is the contents of a sample file made by selecting localhost, and
> accepting the defaults (at least, as I have them configured):
Please correct my comments if they're wrong.
> %ArchiveReq = (
> 'archiveloc' => '/var/lib/BackupPC/removable',
That's where the archive goes. Can be either a special file (eg. tape,
partition) or a directory. -o option for my script (default
/var/lib/backuppc/removable, because I use Debian ;-).
> 'reqTime' => '1170042557',
Time the request file is generated (Perl: time()). Not important except for
reference. Actually, since the request file is interpreted as Perl code, you
*could* write
reqTime => time (),
in a 'static' request file instead. That would be evaluated whenever the
request file is read, which may or may not be better than leaving a fixed
value. I substitute the value according to when my script is run.
> 'BackupList' => [
> '12'
> ],
List of backup numbers to archive. Corresponds element by element with
HostList below, i.e. for archiving backup 1 on host1 and backup 2 on host2
you'd have BackupList = [1, 2] and HostList = ['host1', 'host2'].
-n option for my script, eg. -n 1,2 (default -1 = last backup).
I don't know how meaningful more than one entry is, but it *is* an array,
and BackupPC::Xfer::Archive loops over them, so ...
> 'host' => 'archive',
Name of archive host. -a option for my script.
> 'parfile' => '5',
> 'archtype' => '0',
> 'compression' => '/bin/cat',
> 'compext' => '.raw',
I left those as they are and otherwise ignored them. Adding options to make
these variable is left as an exercise to the reader ;-).
> 'HostList' => [
> 'localhost'
> ],
List of host names to archive. See 'BackupList'. -h option for my script.
> 'user' => 'backupuser',
I ignored that, except for providing a -u option defaulting to 'backuppc'
(and using that for the '<user>' parameter as well).
> 'splitsize' => '0000000'
Ignored. Is that an octal zero, by the way?
> );
Looks sad, but it's just the closing brace of the list assigned to the hash
%ArchiveReq and the semicolon ending the statement. (;
> How Your Code Can Submit an Archive
> ===================================
>
> [...]
>
> There are two problems with this. The first is that the specific backup
> number to be archvied (BackupList) is hard-coded.
Solved by providing a command line option, verifying existance of the specified
backup beforehand (or at least trying to) and changing negative values to the
corresponding backup number for your convenience :).
> The second problem is the fact that the time the job was requested
> (reqTime) is also hard-coded.
Also solved.
> Once you have created the request file in the proper directory, you
> simply tell BackupPC to process it. That can be done easily from the
> command line:
>
> __INSTALLDIR__/bin/BackupPC_serverMesg archive <user> <archiveclient>
> <reqFileName>
You'll need to check and probably change the two paths in the 'use lib' and
'use constant' lines.
Does anyone know whether the <user> parameter has to be a "valid" user, as
far as BackupPC is concerned?
> This will run your job just as if you had submitted it from the GUI. It
> can be scripted from wherever you would like, such as from a crontab,
> just as easily as using BackupPC_archiveHost.
Note that you'll need permissions on the pc directories, so you're probably
best off to run it as backuppc user. If you do at all, that is. Disclaimer:
USE AT YOUR OWN RISK. It's like IP: best effort, no guarantees :).
> Notes about Archive Parameters
> ==============================
>
> I have not delved extensively into the code to find out exactly how each
> parameter is parsed, and what (if any) error-checking is done.
Neither have I.
> ** First, why is there a user parameter on the command line, when the
> request file also contains this information? How are they used?
I decided to supply the same user name in both places.
> ** Second, how is reqTime used?
I'd guess for generating correct output about when the job was submitted
via the CGI interface. Mightn't it run arbitrarily later than submitted due
to MaxBackups/MaxUserBackups?
> ** Third: how does BackupPC_archive know which number is the latest
> backup? Is there an easy way to do this outside of BackupPC, preferably
> without having to duplicate code?
You don't need to "do it outside of BackupPC". You can use the library, just
like BackupPC does. That's what it's there for (and that's what I wanted to
play around with).
> The GUI code that determines the latest backup uses
> $bpc->BackupInfoRead($host) to get that information. That's how the
> latest backup is determined by the GUI.
>
> I would like to know what the code that parses the archive request
> file's backup number parameter does, but it's parsed by $xfer, and I'm
> not sure where that code is. I think that this is merely passed to
> BackupPC_archiveHost, but I am not sure.
Correct, via $Conf {ArchiveClientCmd}.
> [On BackupPC::Lib::BackupInfoRead ()]
> It parses __TOPDIR__/pc/$host/backups and gets the info from there.
> Others have suggested parsing the filesystem to get this information,
> but I am not confident of its robustness. It *should* work reasonably
> well, but obviously the above way is safer.
You've got redundant information: the filesystem and the backups file. If
the supposedly identical information doesn't match, it won't work. Either way,
since both are needed. You *could*, after all, delete the directories and keep
the backups file (or restore it from backup or copy it from somewhere else or
append bogus information or ...). In that respect, the directory contents
are the more definitive indication of what information is actually present.
> So, I would like to figure out a way to do one of these two things:
> either get the latest backup number or save archive files with the
> proper number in the filename even when using "-1" as the backup number.
See the script for how I get the backup number. Actually, see your posting,
since it's clearer there without my loop over potentially more than one host
and the existance tests.
After having finished a first attempt and gotten it deleted by
{stupidity,lack of sleep,too fast fingers}, my second attempt is less
commented and slightly less tested. Let's say, I got as far as getting
error messages where I should (and tried) and a
Got reply: archive error: unknown archive host archive
which is correct for my setup at the moment. The request file generated
looks good enough for me (and the Perl interpreter).
Regards,
Holger
#!/usr/bin/perl -T
#
# Quick hack to submit an archive request to the BackupPC daemon from the
# command line. Note that this script probably needs to be run as user
# 'backuppc' (or whatever the BackupPC user is on your system) in order to
# be able to view the pool.
# This is meant as a proof of concept, not as an example for beautiful Perl
# code. USE AT OWN RISK!
# Note that excerpts of the code are taken from BackupPC via a discussion
# on the backuppc-users mailing list, so this code is probably GPL too. Can't
# hurt in any case :).
# Hacked together by Holger Parplies after detailed BackupPC code analysis by
# Timothy J. Massey.
# Please change the two paths in the next block to match your setup.
use strict;
use lib '/usr/share/backuppc/lib'; # Debian; change to fit your needs
use BackupPC::Lib;
use Getopt::Std;
use constant BACKUPPCBINDIR => '/usr/share/backuppc/bin'; # Debian; change ...
# All changes required below here are due to bugs :).
$ENV {PATH} = '/bin:/usr/bin'; # for taint checking / system (which doesn't
# use the path)
my %opts = (
a => 'archive', # archive host
h => 'localhost', # list of hosts to archive
n => '-1', # list of backup numbers to archive
o => '/var/lib/backuppc/removable', # output destination
u => 'backuppc', # BackupPC user to run the archive command as
# Adding options, part 1: add X => 'default value' here
);
# Adding options, part 2: add X: in the first argument to getopts() and,
# preferably, the usage note
unless (getopts ('a:h:n:o:u:', \%opts)) {
die "Usage: $0 [-a archivehost] [-h hostlist] [-n numlist] [-o outloc] [-u
user]\n";
# I *had* a more descriptive message, before writing to a group writable
# setuid script deleted my first attempt. *sigh*
}
# some variables
my @hosts = split /[:,\/]/, $opts {h}; # array of hosts
my @nums = split /[:,\/]/, $opts {n}; # array of backup numbers
my $hostlist; # textual list of hosts
my $numlist; # textual list of backup numbers
my $reqtime = time; # timestamp
my $bpc = new BackupPC::Lib
or die "Can't create BackupPC object!\n";
my $topdir = $bpc -> {TopDir}; # shorthand for TopDir
my $rfn; # request file name
my $frfn; # full request file name (with path)
my %backups; # backup info indexed by host name
# clone host or backup number if appropriate
@hosts = map { $hosts [0] } (1 .. @nums)
if @hosts == 1 and @nums > 1;
@nums = map { $nums [0] } (1 .. @hosts)
if @nums == 1 and @hosts > 1;
# complain if params are unmatched or missing
if (@hosts != @nums or @hosts == 0) {
die "Hey, you need matching host and backup number specifications or only\n"
. "one of either!\n";
}
# translate backup numbers if negative, check for existing hosts and backups
for (my $i = 0; $i < @nums; $i ++) {
if (not -d "$topdir/pc/$hosts[$i]") {
die "Host $hosts[$i] does not exist!\n";
}
$backups {$hosts [$i]} = [ $bpc -> BackupInfoRead ($hosts [$i]) ]
unless exists $backups {$hosts [$i]};
$nums [$i] = $backups {$hosts [$i]} [$nums [$i]] {num}
if $nums [$i] < 0 and -$nums [$i] <= @{$backups {$hosts [$i]}};
if (not grep {$_ -> {num} == $nums [$i]} @{$backups {$hosts [$i]}}) {
die "There is no backup $nums[$i] of host $hosts[$i]!\n";
}
}
$hostlist = '"' . join ('", "', @hosts) . '"';
$numlist = '"' . join ('", "', @nums) . '"';
# untaint archive host name
if ($opts {a} =~ m{^([^/]+)$}) {
$opts {a} = $1;
} else {
die "Archive host name '$opts{a}' invalid!\n";
}
# check whether the archive hosts at least apparently exists
unless (-d "$topdir/pc/$opts{a}") {
die "The archive host $opts{a} does not exist or the pool is corrupt!\n";
}
# create new request file (without thinking too much about security :)
for (my $i = 0; ; $i ++) {
$rfn = "archiveReq.$$.$i"; $frfn = "$topdir/pc/$opts{a}/$rfn";
last
unless -f $frfn;
}
open REQF, ">$frfn"
or die "Can't create request file $frfn: $!\n";
# Adding options, part 3: use $opts{X} presumably in here ...
print REQF <<EOREQ;
\%ArchiveReq = (
'archiveloc' => '$opts{o}',
'reqTime' => '$reqtime',
'BackupList' => [
$numlist
],
'host' => '$opts{a}',
'parfile' => '5',
'archtype' => '0',
'compression' => '/bin/cat',
'compext' => '.raw',
'HostList' => [
$hostlist
],
'user' => $opts {u},
'splitsize' => '0000000'
);
EOREQ
close REQF;
# submit request to the BackupPC daemon
system (BACKUPPCBINDIR . '/BackupPC_serverMesg',
'archive', $opts {u}, $opts {a}, $rfn);
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/
