Re: Best remote backup method?
Also, dump/restore allows you to use snapshots on a live filesystem (I would test it properly on a large FS with heavy activity). But it's worth pointing out that this is fully possibly with any backup tool - just run mksnap_ffs and backup a mounted snapshot. I do this with rdiff-backup for example. Now, if you are worried about backing up the whole filesystem...well, just tell dump not to dump it :) man chflags (in particular, the nodump flag) man dump (in particular, -h ) The problem with this is for me two-fold: (1) It's a global property. I can't take different backups that include/exclude different things. (2) I can't easily express backup /usr/var/db/my-important-database without seting nodump on a bunch of stuff except that. In other words, I want exclude by default, while dump and the chflags system provides include by default. That said I do like dump's integration with snapshots and overall coherent feeling. If backup diskspace and bandwidth was not a concern I'd use it. -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller [EMAIL PROTECTED]' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org signature.asc Description: OpenPGP digital signature
Re: Best remote backup method?
--On Thursday, May 17, 2007 11:30:00 +0200 Peter Schuller [EMAIL PROTECTED] wrote: Also, dump/restore allows you to use snapshots on a live filesystem (I would test it properly on a large FS with heavy activity). But it's worth pointing out that this is fully possibly with any backup tool - just run mksnap_ffs and backup a mounted snapshot. I do this with rdiff-backup for example. Now, if you are worried about backing up the whole filesystem...well, just tell dump not to dump it :) man chflags (in particular, the nodump flag) man dump (in particular, -h ) The problem with this is for me two-fold: (1) It's a global property. I can't take different backups that include/exclude different things. (2) I can't easily express backup /usr/var/db/my-important-database without seting nodump on a bunch of stuff except that. In other words, I want exclude by default, while dump and the chflags system provides include by default. That said I do like dump's integration with snapshots and overall coherent feeling. If backup diskspace and bandwidth was not a concern I'd use it. I want to thank everyone who contributed to this thread. You've given me a great deal to think about. I'll be reading over the responses again, carefully, and decide what I think the best answer is. -- Paul Schmehl ([EMAIL PROTECTED]) Senior Information Security Analyst The University of Texas at Dallas http://www.utdallas.edu/ir/security/
Best remote backup method?
I'm presently backing up two servers in a remote location to a usb drive
located elsewhere by using rsync over ssh (all three are FreeBSD boxes.)
After the recent discussion about dump, I'm wondering if I would gain
anything by using dump rather than rsync. Has anyone used both? Any
thoughts as to which is better and why?
The rsync command I use is:
rsync -avz ${LOCALDIR} -e ssh -i ${KEY} ${REMOTEHOST}:${REMOTEDIR}
--
Paul Schmehl ([EMAIL PROTECTED])
Senior Information Security Analyst
The University of Texas at Dallas
http://www.utdallas.edu/ir/security/
Re: Best remote backup method?
On Wed, May 16, 2007 at 01:38:13PM -0500, Paul Schmehl wrote:
I'm presently backing up two servers in a remote location to a usb drive
located elsewhere by using rsync over ssh (all three are FreeBSD boxes.)
After the recent discussion about dump, I'm wondering if I would gain
anything by using dump rather than rsync. Has anyone used both? Any
thoughts as to which is better and why?
The rsync command I use is:
rsync -avz ${LOCALDIR} -e ssh -i ${KEY} ${REMOTEHOST}:${REMOTEDIR}
Well, I suppose you could dump to a file and then rsync it to the
other machine...
Basically, I think dump would give you a file over there that
is in dump format and rsync would give you something that
essentially duplicates the file system you are backing up rather
than creating a dump file.
A dump file would be easier to pull back and restore if you had a
major/catastrophic loss of disk and had to replace the disk (or undo
an inappropriate rm -rf *).
Either would be about the same effort restoring a single or handful
of files from backup.
jerry
--
Paul Schmehl ([EMAIL PROTECTED])
Senior Information Security Analyst
The University of Texas at Dallas
http://www.utdallas.edu/ir/security/
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Best remote backup method?
I'm presently backing up two servers in a remote location to a usb drive
located elsewhere by using rsync over ssh (all three are FreeBSD boxes.)
After the recent discussion about dump, I'm wondering if I would gain
anything by using dump rather than rsync. Has anyone used both? Any
thoughts as to which is better and why?
The rsync command I use is:
rsync -avz ${LOCALDIR} -e ssh -i ${KEY} ${REMOTEHOST}:${REMOTEDIR}
Personally I never find dump/restore practical since I seldom want to
backup entire filesystems for performance/diskspace reasons.
I have not found any truly perfect solution; what I am using most often
is rdiff-backup which combines the properties of incremental backups and
rsync. It keeps an up-to-date mirror along with reverse diffs.
The good parts are that:
* It works without hassle right off the bat over ssh. No fuss.
* It is able to backup ownership information without running as root,
because meta-data is stored separately from the files (but at the same
time the up-to-date mirror is a plain tree on disk so you do not
actually have to use rdiff-backup for restores unless you care about
ownership and such).
* Other than using the rsync algorithm for transfers, the actual reverse
diffs are also expressed at a more granular level than entire files. End
result is that a daily backup of that 5 gig log file will not consume 5
gigs of storage per day (but will be very slow to backup).
The main downsides are IMO:
* It's fairly slow. I don't generally see it saturating neither disk nor
networking. Sometimes it's CPU bound, but oftentimes it's just slow
without an obvious bottleneck (probably architectural in the protocol).
* It has some reliability issues. A common problem is that certain
meta-dat is kept in gzip files, and in certain cases of rdiff-backup
being interrupted these files will get corrupted and rdiff-backup will
refuse to function due to the gzip library throwing an exception.
* While it basically works like rsync with history and is thus very
simple to get started with, it does just that. If you want things like
automatic rotation schemes with hourly/daily/etc you have to script that
on top.
--
/ Peter Schuller
PGP userID: 0xE9758B7D or 'Peter Schuller [EMAIL PROTECTED]'
Key retrieval: Send an E-Mail to [EMAIL PROTECTED]
E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org
signature.asc
Description: OpenPGP digital signature
Re: Best remote backup method?
On Wed, May 16, 2007 at 01:38:13PM -0500, Paul Schmehl wrote:
I'm presently backing up two servers in a remote location to a usb drive
located elsewhere by using rsync over ssh (all three are FreeBSD boxes.)
After the recent discussion about dump, I'm wondering if I would gain
anything by using dump rather than rsync. Has anyone used both? Any
thoughts as to which is better and why?
The rsync command I use is:
rsync -avz ${LOCALDIR} -e ssh -i ${KEY} ${REMOTEHOST}:${REMOTEDIR}
With dumps it is easier to keep different ones around. If you rsync a
directory, all previous changes are lost. If you rsync to a different
directory every time to keep different versions, you might as well use
tar, because rsync won't save a lot of space/time in that case. And dump
will backup all ufs2 features such as flags and acls. I'm not sure if
rsync can manage that. It's also easy to compress dumps, which can save
a lot of space.
But if you need to lift a single file from a backup, it might be easier
with rsync, although dump has an interactive mode to select stuff to
restore as well.
A compelling reason to use rsync would be if the file system that is to
be backed up is so large that more than one backup won't fit on your
backup disk anyway. In that case rsync can save you a lot of time.
Roland
--
R.F.Smith http://www.xs4all.nl/~rsmith/
[plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated]
pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725)
pgpziGdV2pUCD.pgp
Description: PGP signature
Re: Best remote backup method?
On Wed, 16 May 2007, Roland Smith wrote:
On Wed, May 16, 2007 at 01:38:13PM -0500, Paul Schmehl wrote:
I'm presently backing up two servers in a remote location to a usb drive
located elsewhere by using rsync over ssh (all three are FreeBSD boxes.)
After the recent discussion about dump, I'm wondering if I would gain
anything by using dump rather than rsync. Has anyone used both? Any
thoughts as to which is better and why?
The rsync command I use is:
rsync -avz ${LOCALDIR} -e ssh -i ${KEY} ${REMOTEHOST}:${REMOTEDIR}
With dumps it is easier to keep different ones around. If you rsync a
directory, all previous changes are lost. If you rsync to a different
directory every time to keep different versions, you might as well use
tar, because rsync won't save a lot of space/time in that case. And dump
will backup all ufs2 features such as flags and acls. I'm not sure if
rsync can manage that. It's also easy to compress dumps, which can save
a lot of space.
Tar is expensive time-wise anyhow after a while if you use compression.
Also, rsync does diffs on files, which can become expensive in terms of time.
-Garrett
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Best remote backup method?
Is there a free NDMP tool for Freebsd?
On 5/16/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:
On Wed, 16 May 2007, Roland Smith wrote:
On Wed, May 16, 2007 at 01:38:13PM -0500, Paul Schmehl wrote:
I'm presently backing up two servers in a remote location to a usb
drive
located elsewhere by using rsync over ssh (all three are FreeBSD
boxes.)
After the recent discussion about dump, I'm wondering if I would gain
anything by using dump rather than rsync. Has anyone used both? Any
thoughts as to which is better and why?
The rsync command I use is:
rsync -avz ${LOCALDIR} -e ssh -i ${KEY} ${REMOTEHOST}:${REMOTEDIR}
With dumps it is easier to keep different ones around. If you rsync a
directory, all previous changes are lost. If you rsync to a different
directory every time to keep different versions, you might as well use
tar, because rsync won't save a lot of space/time in that case. And dump
will backup all ufs2 features such as flags and acls. I'm not sure if
rsync can manage that. It's also easy to compress dumps, which can save
a lot of space.
Tar is expensive time-wise anyhow after a while if you use compression.
Also, rsync does diffs on files, which can become expensive in terms of
time.
-Garrett
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to
[EMAIL PROTECTED]
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Best remote backup method?
On Wed, May 16, 2007 at 10:27:35PM +0200, Roland Smith wrote:
On Wed, May 16, 2007 at 01:38:13PM -0500, Paul Schmehl wrote:
I'm presently backing up two servers in a remote location to a usb drive
located elsewhere by using rsync over ssh (all three are FreeBSD boxes.)
After the recent discussion about dump, I'm wondering if I would gain
anything by using dump rather than rsync. Has anyone used both? Any
thoughts as to which is better and why?
The rsync command I use is:
rsync -avz ${LOCALDIR} -e ssh -i ${KEY} ${REMOTEHOST}:${REMOTEDIR}
With dumps it is easier to keep different ones around. If you rsync a
directory, all previous changes are lost. If you rsync to a different
directory every time to keep different versions, you might as well use
tar, because rsync won't save a lot of space/time in that case. And dump
will backup all ufs2 features such as flags and acls. I'm not sure if
rsync can manage that. It's also easy to compress dumps, which can save
a lot of space.
But if you need to lift a single file from a backup, it might be easier
with rsync, although dump has an interactive mode to select stuff to
restore as well.
A compelling reason to use rsync would be if the file system that is to
be backed up is so large that more than one backup won't fit on your
backup disk anyway. In that case rsync can save you a lot of time.
And, if you _really_ screw things up, like 'rm -rf foo *'
instead of 'rm -rf foo*' from /usr/bin, bunzip2 and restore are right there
in /rescue, while rsync isn't. And getting rsync to work when /usr/bin is
hosed is quite a lot of work (no compiler etc).
And yes, these things happen (speaking from personal experience). :-(
So making backups with something that is available in /rescue or on the
boot CD is definitely a huge plus. Because if you need those backups,
chances are you need them badly.
Roland
--
R.F.Smith http://www.xs4all.nl/~rsmith/
[plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated]
pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725)
pgpiwml5xnL8X.pgp
Description: PGP signature
Re: Best remote backup method?
On Wed, May 16, 2007 at 10:15:05PM +0200, Peter Schuller wrote:
I'm presently backing up two servers in a remote location to a usb drive
located elsewhere by using rsync over ssh (all three are FreeBSD boxes.)
After the recent discussion about dump, I'm wondering if I would gain
anything by using dump rather than rsync. Has anyone used both? Any
thoughts as to which is better and why?
The rsync command I use is:
rsync -avz ${LOCALDIR} -e ssh -i ${KEY} ${REMOTEHOST}:${REMOTEDIR}
Personally I never find dump/restore practical since I seldom want to
backup entire filesystems for performance/diskspace reasons.
That is one reason I divide things up in to more partitions
and thus more specific filesystems. I can isolate the things I
want to back up regularly from those that do not need it.
jerry
.
automatic rotation schemes with hourly/daily/etc you have to script that
on top.
--
/ Peter Schuller
PGP userID: 0xE9758B7D or 'Peter Schuller [EMAIL PROTECTED]'
Key retrieval: Send an E-Mail to [EMAIL PROTECTED]
E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Best remote backup method?
On Wed, 16 May 2007 22:55:04 +0200
Roland Smith [EMAIL PROTECTED] wrote:
And, if you _really_ screw things up, like 'rm -rf foo *'
instead of 'rm -rf foo*' from /usr/bin, bunzip2 and restore are right there
in /rescue, while rsync isn't. And getting rsync to work when /usr/bin is
hosed is quite a lot of work (no compiler etc).
And yes, these things happen (speaking from personal experience). :-(
So making backups with something that is available in /rescue or on the
boot CD is definitely a huge plus. Because if you need those backups,
chances are you need them badly.
Very true.
Also, dump/restore allows you to use snapshots on a live filesystem (I would
test it properly on a large FS with heavy activity).
Now, if you are worried about backing up the whole filesystem...well, just
tell dump not to dump it :)
man chflags (in particular, the nodump flag)
man dump (in particular, -h )
having said that, each tool has its advantages i use rdiff-backup for my
laptop, but dump/restore on servers .
_
{Beto|Norberto|Numard} Meijome
If you were supposed to understand it, we wouldn't call it 'code'.
I speak for myself, not my employer. Contents may be hot. Slippery when wet.
Reading disclaimers makes you go blind. Writing them is worse. You have been
Warned.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
