On Thu, 30 Apr 2009 08:25:38 +0100 Neil Bothwick <[email protected]> wrote:
> What backup medium are you using?
Oh my, I've managed to forget about it!
The medium is regular sata2 hard drives with ext3 filesystem on a
dedicated backup machine with quite rusty debian (etch) linux.
Most backed-up systems (that I care about) are actually freebsd 6, the
rest are linux. Most stored backups are 20-60 GB.
Main bottleneck here is the network - quite laggy 100 Mbps link,
because this backup server is quite far and isolated from the rest.
Also it's completely inaccessible from backed-up machines, aside from
reverse tunnels, which I rarely use as a dirty hacks.
And this link has tendency to go down every once in a while,
interrupting ongoing transfers.
That said, nightly backup should always be available, so the backups
are actually created on hotswap sata2 drives of each individual
machine and grabbed by backup server over ssh and, in some cases, nfs.
These days the scripts on the backup server quite frequently (10-50
times a day) connect to the other hosts and receive requests for
certain paths from stored backups. So they parse backups with tar
picking out given paths and pushes them back, as requested.
Needless to say, it is slow, hence the need.
Well, that's probably a bit more verbose than necessary, but the point
is that (I believe) the backups should be created right on backed-up
systems' hard disks, so:
1. Have random access to backup storage.
2. Prolonged io/cpu load is a bad thing.
3. Compression (at least of gzip ratio) is a must, because of limited
storage on backed-up machines.
4. In-backup seek times should be lower than tar (which is scanning
the whole file).
5. I write py scripts for a living, so the question is really in a
backup format - transfer and storage structure is not the issue.
Easiest thing I've thought of is just to generate "tar index" on first
archive pass and then just skip to the recorded point in ungzipped
stream, feeding the rest to tar, stopping when necessary, but there's
no point to debug and maintain this system if there are better
solutions already.
> If hard disks, do you have a separate machine for storing them? If
> so, BackupPC may suit your needs. The ebuild in Portage is out of
> date but the latest version is available from Bugzilla -
> http://bugs.gentoo.org/show_bug.cgi?id=141018
>
> It allows restoration of individual files or directories, from any
> backup point, and doesn't require any special software on the
> machines being backed up, only SSH access. It can backup Windows and
> *nix machines.
Thanks, will check it out, but I'm afraid that live network backups
aren't the best solution in my case.
--
Mike Kazantsev // fraggod.net
signature.asc
Description: PGP signature
