Hi,
Les Mikesell wrote on 26.01.2007 at 20:53:11 [Re: [BackupPC-users] Long: How
BackupPC handles pooling, and how transfer methods affect bandwidth usage]:
> Timothy J. Massey wrote:
> > In reality, I'm still talking about a
> > custom BackupPC client, but instead of targeting the host, I'm targeting
> > a much more confined target: BackupPC servers. How about BackupPC
> > servers that can replicate their data to each other in remote locations?
> > A local BackupPC server is set up in the default way: incrementals are
> > performed daily, fulls are performed weekly. In addition, this server
> > also replicates all of its backups once a week to a remote BackupPC
> > server, which integrates them into its pool.
> >
> > There are a lot of advantages to this. It can take advantage of pooling
> > *before* an item is sent.
>
> But what is the advantage over just letting the remote server make its
> run directly against the same targets?
I think most of these ideas were in Tim's original post:
1.) Fast local backups
You get backups of your client machines done without errors caused by
long backup time over slow connections (such as data inconsistencies
as well as the usual timeout and connection loss problems).
Laptop users can initiate a backup whenever they're in the office, even
if they don't have as much time as a remote backup would take.
2.) Offsite storage over network
You also get your backups mirrored to an offsite location *eventually*,
regardless of intermittent network problems. If your link goes down for
a week, there's no way to get your backup history for that week after
the fact with only your target machines and the remote BackupPC server.
If you have a local BackupPC server that has kept track, there's
virtually no difference when you transfer the backups offsite, as long
as you do at all in the end.
If your local BackupPC server fails and the link doesn't, you might even
be able to temporarily switch backups to the offsite server until you get
the replacement up and running, and then replicate the history back.
3.) Reduced bandwidth requirement
Because both ends understand the pooling mechanism, you have to transfer
identical files at most once, and that only if they are not in the offsite
pool yet. You are not limited to the "previous backup" comparison, any
identical file in the pool will do to save you the transfer.
That also means you could cut down your bandwidth usage by occasionally
carrying a DVD full of files to the offsite server and adding them to the
pool there (eg. by backing them up with BackupPC). No need to worry about
filesystem layout or anything - the pure fact that they exist saves you
the transfer (they have to end up in the correct place in the pool, of
course, but BackupPC will take care of that. If your file was named
/usr/src/foo.tar, backing it up from /mnt/cdrom/bar.tar will do fine,
though I'm not sure why you'd rename the tar file ;-).
4.) Reduced computing time/disk IO
Because the transfer process knows beforehand that any set of files is
identical, no further comparisons (involving CPU time and disk IO (= disk
wear)) beyond the first are necessary for this set.
If you trust the rsync digest stored in the pool file as a result of
having checksum-seeding turned on (on both sides), comparing two pool
files becomes really cheap.
These savings do not apply to the client machines if you're comparing
with *only* a remote backup and no local one. For them, it's one regular
BackupPC backup either way. They do however take away CPU load and disk
wear from your ("central"?) offsite BackupPC server, moving it to the
local BackupPC server.
And if you compare it with *two* backups, you're obviously saving the
client machines one of them (in addition to taking load off your offsite
server).
5.) Reduced local hardware requirements
You could keep a short local backup history to which the users have fast
access and a long offsite history (seven years? :) to which no immediate
user access is necessary (or even possible - depending on your
requirements).
6.) Flexible network topology
For a remote server to backup local targets, it would need to be able to
address them, and it would need credentials to access the files (ssh
key, rsyncd password, whatever). With a replicating BackupPC server, it
would be possible to either limit access (IP-wise and credential-wise)
to this server, or, alternatively, to have it initiate the connection
(reversing the visibility and credentials question).
True, you can do some of that with a VPN too, but there may be valid
reasons not to allow your offsite location to access your network. Or to
decide which files are backed up to it and which aren't.
7.) Simplicity of migrating hosts
I think that was the initial thought :-).
I think "a lot of advantages" is an understatement. It would allow a lot of
complexity. I'm not saying it would be simple to implement, but my guess would
be that it should be feasible without even touching existing code.
I've heard "BackupPCd" mentioned several times lately. It sounded like an
upcoming backup-client-side application serving as an endpoint for an
alternative transfer method to rsync(d)/tar/smb. Is that correct?
Some of that would probably apply there, and such a transfer method might
conversely be a place to "plug in" such a replication mechanism as envisioned
by Tim.
Regards,
Holger
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
BackupPC-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/
