Re: backing up freebsd
2011-07-03 13:56, Tim Dunphy skrev: hello list!! Hello Tim. Could someone please provide a tip on how I can go about backing up the FreeBSD client? http://www.se.freebsd.org/doc/en_US.ISO8859-1/books/handbook/backup-basics.html ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: Backing up freebsd to 1 file?
-Original Message-
From: owner-freebsd-questi...@freebsd.org [mailto:owner-freebsd-
questi...@freebsd.org] On Behalf Of J.D. Bronson
Sent: Sunday, April 18, 2010 3:23 PM
To: freebsd-questions@freebsd.org
Subject: Backing up freebsd to 1 file?
I have a freebsd 8.0 install and was wondering if it is possible to tar
up the entire install...for backup purposes.
# cd /
# tar -cvf backup.tar {list of directories}
then I can ftp the tar file out to another machine.
This works in theory, but if I need to do a restore tar complains
on 'tar -xpf backup.tar'.
Under OpenBSD, this works as expected. It has given me an easy way
to backup/move/restore or anything I want to do w/o complaining.
I am running Freebsd on a machine that has no other drives/tapes or
anything so my options for backup are limited.
All I am trying to do is get a complete image (or snapshot) of my
entire
install on this machine and then if I needed to reload or reinstall, I
could do a bare bones freebsd install, copy over the tar'd up file and
extract it from within / and then reboot an I would be go to go.
Thoughts on this would be appreciated...
Perhaps http://ra.phid.ae/stuff/mm-backup-0.9.sh.txt has something that you
like.
--
Regards,
T. Koeman, MTh/BSc/BPsy; Technical Monk
MediaMonks B.V. (www.mediamonks.com)
Please quote all replies in correspondence.
smime.p7s
Description: S/MIME cryptographic signature
Re: Backing up freebsd to 1 file?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 19/04/2010 06:52:29, Sergio de Almeida Lenzi wrote: but I use zfs and I think that during shutdown, /etc/rc.d/zfs is called stop so it unmounts all zfs partition... (I did not tested...)... so It must be called /etc/rc.d/zfs start again... (just a few inconvenient...) That's actually a very interesting point, and as far as I can tell from looking at the rc scripts, what you describe is exactly what would happen. Now wondering how a pure-ZFS system manages going to single user. Cheers, Matthew - -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.14 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkvL/s4ACgkQ8Mjk52CukIzoUQCfYFhoGDPr4bN3SO9PXLp6U54W 6moAn1fxKG+7Xp+76xNwzUdF6mAGxd9u =RgU6 -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
Sergio == Sergio de Almeida Lenzi lenzi.ser...@gmail.com writes: It kills everything ungracefully and will screw up anything that needs to sync state to disk -- like mysql. Just use shutdown(8): it's what it's there for. # shutdown now Going single user to make backups Cheers, Matthew Sergio Ok you are right... Sergio for me worked because I never use mysql... Sergio but I use zfs and I think that during shutdown, /etc/rc.d/zfs is Sergio called stop Sergio so it unmounts all zfs partition... (I did not tested...)... Nope. shutdown doesn't appear in /etc/rc.d/zfs keywords, so it won't get stop during normal shutdown. That must happen later. -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 mer...@stonehenge.com URL:http://www.stonehenge.com/merlyn/ Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc. See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 19/04/2010 16:16:21, Randal L. Schwartz wrote: Nope. shutdown doesn't appear in /etc/rc.d/zfs keywords, so it won't get stop during normal shutdown. That must happen later. Dammit. I know this really -- but for some reason i had it in my head that the keyword was 'noshutdown' with exactly the opposite semantics. You're entirely correct. D'Oh! Matthew - -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.14 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkvMjroACgkQ8Mjk52CukIz1xwCfQSfxlb1f14A/dxwlemUw0rV9 gtIAmwR4rM7Xge3PlzKEW46hipRMpOhV =5SjO -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
On 18 April 2010 15:56, J.D. Bronson jd_bron...@sbcglobal.net wrote: be created by the time your system boots on. Nice answer by Sergio, but I personally would use the j option with tar to compress to bzip2; 3) tar --one-file-system -cvjf /mnt/backup.tbz ./ var usr home Though I prefer personally to use dump/restore because: - If you're on UFS, you don't have to single-user the system, just use the L option (live filesystem) - Restore has an awesome 'interactive' mode - See Zwicky [1] I'll send you my dump scripts if you're interested. It's dead easy to use! Chris [1] http://www.coredumps.de/doc/dump/zwicky/testdump.doc.html . I think Sergio has a nice script. I had been doing something similar but I know I recall when untarring (restoring if you will) it was complaining about not being able to do things. It was not sockets and similar stuff that gets rebuilt on reboot. I do not have failures handy to post (yet). Truth be told? - I am running FreeBSD hosts within ESXi. I can backup the hosts within ESXi but need to take the host offline and its a cumbersome ordeal. If I had RAID on ESXi, I wouldn't be so worried per se but this is not an option. ESXi is very fussy about what is supported and I dont have the $ for SCSI and SCSI Raid. Basically what I need to do is create a fully restorable backup for 2 reasons: 1. Easy to create another host on ESXi. I can setup/flavor my fbsd install and then once thats done, setup another host. 2. Obvious backup reasons. ...right now, if the SATA drive fails that is hosting the fbsd install I am dead in the water. I have 5 hosts on this machine spread across 4 SATA drives but nothing is mirrored or RAIDed in anyway. I am at the mercy of these drives w/o any backup- Yeah, use dump. It's excellent, and you can bz2 the results. My script for dumping: #!/bin/sh # $Id: backuphdd.sh,v 1.3 2010/02/02 13:02:06 root Exp $ # $Log: backuphdd.sh,v $ # Revision 1.3 2010/02/02 13:02:06 root # Changed so that backup/spare is only manipulated when backup level is 0 # # Revision 1.2 2009/12/22 16:13:05 root # Now uses bzip2 LEVEL=$1 mount /backup/dumps mv /backup/dumps/root_level_$LEVEL.bz2 /backup/dumps/root_level_$LEVEL.bz2.old dump -$LEVEL -Lauf - / | bzip2 /backup/dumps/root_level_$LEVEL.bz2 mv /backup/dumps/var_level_$LEVEL.bz2 /backup/dumps/var_level_$LEVEL.bz2.old dump -$LEVEL -Lauf - /var | bzip2 /backup/dumps/var_level_$LEVEL.bz2 mv /backup/dumps/usr_level_$LEVEL.bz2 /backup/dumps/usr_level_$LEVEL.bz2.old dump -$LEVEL -Lauf - /usr | bzip2 /backup/dumps/usr_level_$LEVEL.bz2 umount /backup/dumps ---end I call it from cron ~3 in the morning with a tower of hanoi rotation; it takes the argument to the script as the dump level; /root/backuphdd.sh 0 performs a level 0 dump of all the drives. Don't forget to back up _all_ your partitions! Dump only backs up separate partitions... Chris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Backing up freebsd to 1 file?
I have a freebsd 8.0 install and was wondering if it is possible to tar
up the entire install...for backup purposes.
# cd /
# tar -cvf backup.tar {list of directories}
then I can ftp the tar file out to another machine.
This works in theory, but if I need to do a restore tar complains
on 'tar -xpf backup.tar'.
Under OpenBSD, this works as expected. It has given me an easy way
to backup/move/restore or anything I want to do w/o complaining.
I am running Freebsd on a machine that has no other drives/tapes or
anything so my options for backup are limited.
All I am trying to do is get a complete image (or snapshot) of my entire
install on this machine and then if I needed to reload or reinstall, I
could do a bare bones freebsd install, copy over the tar'd up file and
extract it from within / and then reboot an I would be go to go.
Thoughts on this would be appreciated...
--
J.D. Bronson
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
On Sun, 18 Apr 2010 08:23:12 -0500, J.D. Bronson jd_bron...@sbcglobal.net
wrote:
I have a freebsd 8.0 install and was wondering if it is possible to tar
up the entire install...for backup purposes.
# cd /
# tar -cvf backup.tar {list of directories}
then I can ftp the tar file out to another machine.
This works in theory, but if I need to do a restore tar complains
on 'tar -xpf backup.tar'.
In this case, you're better using dump partition-wise, or
just use dd to copy the whole disk - this may lead to large
files, so adding compression is often useful.
Under OpenBSD, this works as expected. It has given me an easy way
to backup/move/restore or anything I want to do w/o complaining.
What exact complains are output by FreeBSD's tar?
I am running Freebsd on a machine that has no other drives/tapes or
anything so my options for backup are limited.
Note that dumping / restoring (especially restoring) is more
easy to be done by booting from a live system (e. g. via CD,
DVD or USB).
All I am trying to do is get a complete image (or snapshot) of my entire
install on this machine and then if I needed to reload or reinstall, I
could do a bare bones freebsd install, copy over the tar'd up file and
extract it from within / and then reboot an I would be go to go.
Well... tar is not so good suited for that. There are things
at file system level that are important to the system, but are
not honored by the tar utility. In this case, dump + restore
provide excellent means for what you're intending. In case of
a failure, use a FreeBSD boot medium with sysinstall or sade
to prepare the disk (slice, partition, newfs), then restore
the dump files to the partitions, reboot, and it's done.
Of course, dd provides an exact 1:1 copy, and you can choose
to copy partitions, slices, or a whole disk. The dump and
restore programs operate on file systems (partitions), while
tar operates on files.
Thoughts on this would be appreciated...
There are some threads in the archives about how to backup
or clone a whole system. You'll find some more inspirations
and considerations there.
--
Polytropon
Magdeburg, Germany
Happy FreeBSD user since 4.0
Andra moi ennepe, Mousa, ...
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
I am very happy with the folowing Supose that you have mount ANOTHER device on /mnt 1) mount /dev/ /mnt 2) init 1 (this closes all applications and drop into single user) 3) tar --one-file-system -cvzf /mnt/backup.tar.gz ./ var usr home 4) umount /mnt 5) exit (reboot from single user to normal operation) === on restore... supose you install a FBSD minimal from the CD/usb. 1) mount /dev/ /mnt 2) tar -xpvf /mnt/backup.tar.gz -C / 3) umount /mnt ===you have restored your system= may be some files (sockets...) are not restored but no problem as they will be created by the time your system boots on. Sergio ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
you can migrate to zfs and then create snapshot of whole disk, import
this snapshot (e.g. via ssh) and then restore it back.
Good luck.
--
Jan Hlodan
On Sun, Apr 18, 2010 at 3:23 PM, J.D. Bronson jd_bron...@sbcglobal.net wrote:
I have a freebsd 8.0 install and was wondering if it is possible to tar up
the entire install...for backup purposes.
# cd /
# tar -cvf backup.tar {list of directories}
then I can ftp the tar file out to another machine.
This works in theory, but if I need to do a restore tar complains
on 'tar -xpf backup.tar'.
Under OpenBSD, this works as expected. It has given me an easy way
to backup/move/restore or anything I want to do w/o complaining.
I am running Freebsd on a machine that has no other drives/tapes or anything
so my options for backup are limited.
All I am trying to do is get a complete image (or snapshot) of my entire
install on this machine and then if I needed to reload or reinstall, I could
do a bare bones freebsd install, copy over the tar'd up file and extract it
from within / and then reboot an I would be go to go.
Thoughts on this would be appreciated...
--
J.D. Bronson
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
5
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
On 18 April 2010 15:37, Sergio de Almeida Lenzi lenzi.ser...@gmail.com wrote: I am very happy with the folowing Supose that you have mount ANOTHER device on /mnt 1) mount /dev/ /mnt 2) init 1 (this closes all applications and drop into single user) 3) tar --one-file-system -cvzf /mnt/backup.tar.gz ./ var usr home 4) umount /mnt 5) exit (reboot from single user to normal operation) === on restore... supose you install a FBSD minimal from the CD/usb. 1) mount /dev/ /mnt 2) tar -xpvf /mnt/backup.tar.gz -C / 3) umount /mnt ===you have restored your system= may be some files (sockets...) are not restored but no problem as they will be created by the time your system boots on. Nice answer by Sergio, but I personally would use the j option with tar to compress to bzip2; 3) tar --one-file-system -cvjf /mnt/backup.tbz ./ var usr home Though I prefer personally to use dump/restore because: - If you're on UFS, you don't have to single-user the system, just use the L option (live filesystem) - Restore has an awesome 'interactive' mode - See Zwicky [1] I'll send you my dump scripts if you're interested. It's dead easy to use! Chris [1] http://www.coredumps.de/doc/dump/zwicky/testdump.doc.html ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
be created by the time your system boots on. Nice answer by Sergio, but I personally would use the j option with tar to compress to bzip2; 3) tar --one-file-system -cvjf /mnt/backup.tbz ./ var usr home Though I prefer personally to use dump/restore because: - If you're on UFS, you don't have to single-user the system, just use the L option (live filesystem) - Restore has an awesome 'interactive' mode - See Zwicky [1] I'll send you my dump scripts if you're interested. It's dead easy to use! Chris [1] http://www.coredumps.de/doc/dump/zwicky/testdump.doc.html . I think Sergio has a nice script. I had been doing something similar but I know I recall when untarring (restoring if you will) it was complaining about not being able to do things. It was not sockets and similar stuff that gets rebuilt on reboot. I do not have failures handy to post (yet). Truth be told? - I am running FreeBSD hosts within ESXi. I can backup the hosts within ESXi but need to take the host offline and its a cumbersome ordeal. If I had RAID on ESXi, I wouldn't be so worried per se but this is not an option. ESXi is very fussy about what is supported and I dont have the $ for SCSI and SCSI Raid. Basically what I need to do is create a fully restorable backup for 2 reasons: 1. Easy to create another host on ESXi. I can setup/flavor my fbsd install and then once thats done, setup another host. 2. Obvious backup reasons. ...right now, if the SATA drive fails that is hosting the fbsd install I am dead in the water. I have 5 hosts on this machine spread across 4 SATA drives but nothing is mirrored or RAIDed in anyway. I am at the mercy of these drives w/o any backup- -- J.D. Bronson ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 18/04/2010 15:37:03, Sergio de Almeida Lenzi wrote: 2) init 1 (this closes all applications and drop into single user) It kills everything ungracefully and will screw up anything that needs to sync state to disk -- like mysql. Just use shutdown(8): it's what it's there for. # shutdown now Going single user to make backups Cheers, Matthew - -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.14 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkvLJY4ACgkQ8Mjk52CukIwq8QCfcu48NVNcs1SOsbtV+rZ98MeR hWkAniPX6+bOIx0ej3CXpT8gzNiKBUB/ =hQW3 -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 18/04/2010 15:19:32, Jan Hlodan wrote: you can migrate to zfs and then create snapshot of whole disk, import this snapshot (e.g. via ssh) and then restore it back. You can create snapshots with UFS too. It's a good way of getting a reasonably consistent backup without having to shut the whole system down. Cheers, Matthew - -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.14 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkvLJgMACgkQ8Mjk52CukIzreQCeJBZYMx2Zi1QGq0K76mUD39/x /LgAnAx44eaGLZjNP9eOg9HyOyeR7CYO =/mZk -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
On Sun, 18 Apr 2010, J.D. Bronson wrote:
I have a freebsd 8.0 install and was wondering if it is possible to tar up
the entire install...for backup purposes.
# cd /
# tar -cvf backup.tar {list of directories}
then I can ftp the tar file out to another machine.
This works in theory, but if I need to do a restore tar complains
on 'tar -xpf backup.tar'.
As others have mentioned, tar is not well suited for this.
Under OpenBSD, this works as expected. It has given me an easy way
to backup/move/restore or anything I want to do w/o complaining.
I am running Freebsd on a machine that has no other drives/tapes or anything
so my options for backup are limited.
All I am trying to do is get a complete image (or snapshot) of my entire
install on this machine and then if I needed to reload or reinstall, I could
do a bare bones freebsd install, copy over the tar'd up file and extract it
from within / and then reboot an I would be go to go.
If you don't have any other drives, where will the backup file be stored
so it survives a system failure or reinstall?
Thoughts on this would be appreciated...
dump/restore is the standard safe way; you can send it over ssh to back
up to a file on another machine. Sometimes people use dd, which can be
effective if you use some tricks like filling unused space with zero so
compression is effective.
There's another option. I've mentioned clonezilla.org here before as a
way to back up Windows systems; it's fast and only copies used sectors.
Newer beta versions of clonezilla now support UFS directly, so they can
back up FreeBSD disks. This is nice because it also backs up the MBR,
and splits the backup files into 2G increments. It may also be faster
than dump/restore. Note that I only noticed the UFS mode lately and
have only tried it once so far, so no real experience on how safe it is
yet.
-Warren Block * Rapid City, South Dakota USA
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
On 4/18/10 10:39 AM, Warren Block wrote: If you don't have any other drives, where will the backup file be stored so it survives a system failure or reinstall? Thoughts on this would be appreciated... dump/restore is the standard safe way; you can send it over ssh to back up to a file on another machine. Sometimes people use dd, which can be effective if you use some tricks like filling unused space with zero so compression is effective. There's another option. I've mentioned clonezilla.org here before as a way to back up Windows systems; it's fast and only copies used sectors. I would sftp/scp the file over to another unix (or windows via samba) machine I have. Then burn the resulting file to DVD RW media. -- J.D. Bronson ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up freebsd to 1 file?
It kills everything ungracefully and will screw up anything that needs to sync state to disk -- like mysql. Just use shutdown(8): it's what it's there for. # shutdown now Going single user to make backups Cheers, Matthew Ok you are right... for me worked because I never use mysql... but I use zfs and I think that during shutdown, /etc/rc.d/zfs is called stop so it unmounts all zfs partition... (I did not tested...)... so It must be called /etc/rc.d/zfs start again... (just a few inconvenient...) Thanks for the tip Sergio ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up FreeBSD and other Unix systems securely
On Mon, May 18, 2009 at 9:38 PM, Karl Vogel vogelke+u...@pobox.comvogelke%2bu...@pobox.com wrote: On Sun, 17 May 2009 09:12:57 -0700, Kelly Jones kelly.terry.jo...@gmail.com said: K I like this plan because it does versioned backups, and doesn't backup K identical files twice. I dislike it because I lose Mozy's unlimited disk K space. K % Is there software that already does this? I have a 3-Tbyte server running FreeBSD-6.1 that does something very similar. I don't bother with encrypting the filenames or hashes because we control the box, and if I'm not at work, other admins might need to restore something quickly. We have around 3.7 million files from 5 other servers backed up under two 1.5-Tbyte filesystems, /mir01 and /mir02. My setup looks like this: +-mir01 | +-HASH | | +-00 | | | +-00 | | | +-01 ... | | +-01 ... | | +-fe | | +-ff | +-server1 | +-server2 +-mir02 | +-HASH | +-server3 | +-server4 | +-server5 The HASH directories have two levels of subdirectories 00-ff. That's been more than sufficient to keep directories from getting too big; I average around 25 files per directory. I do hourly backups on the other fileservers using something like the find and timestamp method you mentioned, but I ignore 0-length files because they always hash to the same value. The backup directories for the second fileserver look like this for 5 May 2009: +-mir01 | +-server2 | | +-2009 | | | +-0505 | | | | +-070700 | | | | | +-doc (filesystem) | | | | | +-home | | | | +-080700 | | | | | +-doc | | | | | +-home ... | | | | +-190700 | | | | | +-home After the backups are rsynced to the backup server, I find any regular files with only one link, compute the RMD160 hash of the contents, and make a hardlink to the appropriate filename under the HASH directory. People love to make copies of copies of files, so this really cuts down on the disk space used. The hardlinks make it easy to avoid restoring things that aren't what the user had in mind; if a file's been corrupted, I can tell when it happened just by looking at the inode, so I don't restore an earlier version that's also junk. I can also tell if there were duplicates anywhere on the fileserver at the time the user lost the good version; it's a lot faster for them to get a known good copy from somewhere else on the fileserver than it is to restore over the network. The software is just a few scripts to do things like find files with just one link, compute hashes, do hardlinks, etc. I can put up a tarball if anyone's interested. Hello Kelly, I am doing something similar at a company i work for. I would be interested to see your scripts to make a comparison. thanks, v -- Karl Vogel I don't speak for the USAF or my company The best way for the Government to maintain its credit is to pay as it goes-not by resorting to loans, but by keeping out of debt-through an adequate income secured by a system of taxation, external or internal, or both. --Pres. William McKinley's First Inaugural Address, March 4, 1897 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org -- network warrior since 2005 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up FreeBSD and other Unix systems securely
On Sun, May 17, 2009 at 09:12:57AM -0700, Kelly Jones wrote:
I tried using Mozy for backups because they offer unlimited space, but
1) they don't support FreeBSD, 2) they encrypt file contents, but NOT
file names, and 3) they don't do true versioned backups. Easy
workaround for 1): rsync to a Mac/Windows and backup from there, but
2) and 3) are more difficult.
Is there any possibility of using your own media locally - such as
tape or a large USB attached disk?If security is such a primary
concern, I can't see sending the data to that type of offsite thing.
Get a couple of large USB SATAs and use dump(8) to back the stuff up
on them.Write them encrypted if you need.
jerry
My plan:
% Use dd if=/dev/random of=mykey to create a random blowfish key
% Blowfish encrypt mykey with a passphrase only I know. Backup the
encrypted blowfish key to a remote host.
% Keep track of when I last ran the backup program (touch
/some/path/timestamp at start of run) and only backup files that've
been modified more recently (find / -newer /some/path/timestamp).
% To backup foo.txt, first bzip2 it and encrypt w/ my blowfish key.
% Then, take the sha1 hash of the bzip'd/encrypted file, and backup
foo.txt to remotehost:/some/path/{sha1 hash}.
% To avoid too many files in one dir, I may backup
b0d0a7da15d5eb94ac76ac4fd81fe6d4fa8e4593 to
remotehost:/some/path/b0/d0/a7/b0d0a7da15d5eb94ac76ac4fd81fe6d4fa8e4593
for example.
% In an SQLite3 db, record the filename I'm backing up, its
timestamp, and its bzip'd/encrypted hash. Store an encrypted copy of
the db on the remote server.
I like this plan because it does versioned backups, and doesn't backup
identical files twice. I dislike it because I lose Mozy's unlimited
disk space.
Questions:
% Does this plan seem secure and reasonable?
% Will backing up the 0-byte file this way make it easy to guess my
blowfish key?
% Is there software that already does this?
% Can this plan be improved?
% Does anyone offer unlimited space for Unix backups?
(safesnaps.com)
% Any general thoughts/comments on this plan?
--
We're just a Bunch Of Regular Guys, a collective group that's trying
to understand and assimilate technology. We feel that resistance to
new ideas and technology is unwise and ultimately futile.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up FreeBSD and other Unix systems securely
Is there any possibility of using your own media locally - such as tape or a large USB attached disk?If security is such a primary concern, I can't see sending the data to that type of offsite thing. Get a couple of large USB SATAs and use dump(8) to back the stuff up on them.Write them encrypted if you need. I'd have to agree with this... After looking at a lot of options, I ended up building a simple freebsd server and connected it to my main server on a separate ethernet port via a twisted ethernet cable. Thus, the server and backup server had a 'private', high speed connection and I can pump tons of data through that connection without paying my colo provider for that bandwidth. A whole server, rather than a USB drive might be overkill, but its a little more flexible, and I can use the backup server for a DNS server, and a few other things, as well. -- John ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up FreeBSD and other Unix systems securely
On Sun, 17 May 2009 09:12:57 -0700, Kelly Jones kelly.terry.jo...@gmail.com said: K I like this plan because it does versioned backups, and doesn't backup K identical files twice. I dislike it because I lose Mozy's unlimited disk K space. K % Is there software that already does this? I have a 3-Tbyte server running FreeBSD-6.1 that does something very similar. I don't bother with encrypting the filenames or hashes because we control the box, and if I'm not at work, other admins might need to restore something quickly. We have around 3.7 million files from 5 other servers backed up under two 1.5-Tbyte filesystems, /mir01 and /mir02. My setup looks like this: +-mir01 | +-HASH | | +-00 | | | +-00 | | | +-01 ... | | +-01 ... | | +-fe | | +-ff | +-server1 | +-server2 +-mir02 | +-HASH | +-server3 | +-server4 | +-server5 The HASH directories have two levels of subdirectories 00-ff. That's been more than sufficient to keep directories from getting too big; I average around 25 files per directory. I do hourly backups on the other fileservers using something like the find and timestamp method you mentioned, but I ignore 0-length files because they always hash to the same value. The backup directories for the second fileserver look like this for 5 May 2009: +-mir01 | +-server2 | | +-2009 | | | +-0505 | | | | +-070700 | | | | | +-doc (filesystem) | | | | | +-home | | | | +-080700 | | | | | +-doc | | | | | +-home ... | | | | +-190700 | | | | | +-home After the backups are rsynced to the backup server, I find any regular files with only one link, compute the RMD160 hash of the contents, and make a hardlink to the appropriate filename under the HASH directory. People love to make copies of copies of files, so this really cuts down on the disk space used. The hardlinks make it easy to avoid restoring things that aren't what the user had in mind; if a file's been corrupted, I can tell when it happened just by looking at the inode, so I don't restore an earlier version that's also junk. I can also tell if there were duplicates anywhere on the fileserver at the time the user lost the good version; it's a lot faster for them to get a known good copy from somewhere else on the fileserver than it is to restore over the network. The software is just a few scripts to do things like find files with just one link, compute hashes, do hardlinks, etc. I can put up a tarball if anyone's interested. -- Karl Vogel I don't speak for the USAF or my company The best way for the Government to maintain its credit is to pay as it goes-not by resorting to loans, but by keeping out of debt-through an adequate income secured by a system of taxation, external or internal, or both. --Pres. William McKinley's First Inaugural Address, March 4, 1897 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Backing up FreeBSD and other Unix systems securely
I tried using Mozy for backups because they offer unlimited space, but
1) they don't support FreeBSD, 2) they encrypt file contents, but NOT
file names, and 3) they don't do true versioned backups. Easy
workaround for 1): rsync to a Mac/Windows and backup from there, but
2) and 3) are more difficult.
My plan:
% Use dd if=/dev/random of=mykey to create a random blowfish key
% Blowfish encrypt mykey with a passphrase only I know. Backup the
encrypted blowfish key to a remote host.
% Keep track of when I last ran the backup program (touch
/some/path/timestamp at start of run) and only backup files that've
been modified more recently (find / -newer /some/path/timestamp).
% To backup foo.txt, first bzip2 it and encrypt w/ my blowfish key.
% Then, take the sha1 hash of the bzip'd/encrypted file, and backup
foo.txt to remotehost:/some/path/{sha1 hash}.
% To avoid too many files in one dir, I may backup
b0d0a7da15d5eb94ac76ac4fd81fe6d4fa8e4593 to
remotehost:/some/path/b0/d0/a7/b0d0a7da15d5eb94ac76ac4fd81fe6d4fa8e4593
for example.
% In an SQLite3 db, record the filename I'm backing up, its
timestamp, and its bzip'd/encrypted hash. Store an encrypted copy of
the db on the remote server.
I like this plan because it does versioned backups, and doesn't backup
identical files twice. I dislike it because I lose Mozy's unlimited
disk space.
Questions:
% Does this plan seem secure and reasonable?
% Will backing up the 0-byte file this way make it easy to guess my
blowfish key?
% Is there software that already does this?
% Can this plan be improved?
% Does anyone offer unlimited space for Unix backups?
(safesnaps.com)
% Any general thoughts/comments on this plan?
--
We're just a Bunch Of Regular Guys, a collective group that's trying
to understand and assimilate technology. We feel that resistance to
new ideas and technology is unwise and ultimately futile.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backing up FreeBSD and other Unix systems securely
Kelly Jones wrote: I tried using Mozy for backups because they offer unlimited space, but 1) they don't support FreeBSD, 2) they encrypt file contents, but NOT file names, and 3) they don't do true versioned backups. Easy workaround for 1): rsync to a Mac/Windows and backup from there, but 2) and 3) are more difficult. % Is there software that already does this? Take a look at tarsnap. http://www.tarsnap.com/ -- Christian Laursen ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Backing up FREEBSD
Hi This is Arun from Singapore. I basically want to know how to back up files if a computer is already running on FREEBSD. Please help me with this as it is urgent. Thanking you Arun ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Backing up FREEBSD
The handbook is your friend: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/backup-basics.html and most likely: man dump Chad On 12/12/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Hi This is Arun from Singapore. I basically want to know how to back up files if a computer is already running on FREEBSD. Please help me with this as it is urgent. Thanking you Arun ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Backing up FREEBSD
As with what he said. :) Dump and Restore are your friends. Also, in a crunch: tar -z -c -f /path/to/your/backup.tar.gz /filesystem Do that once for each mounted filesystem, and make sure backup.tar.gz lives on another system, perhaps on an nfs mount. Chad Gross wrote: The handbook is your friend: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/backup-basics.html and most likely: man dump Chad On 12/12/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Hi This is Arun from Singapore. I basically want to know how to back up files if a computer is already running on FREEBSD. Please help me with this as it is urgent. Thanking you Arun ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Backing up FREEBSD
On Tue, Dec 12, 2006 at 06:30:09AM -0500, [EMAIL PROTECTED] wrote: Hi This is Arun from Singapore. I basically want to know how to back up files if a computer is already running on FREEBSD. Please help me with this as it is urgent. It depends a little on what media you have available to store a backup. The long time traditional media would be tape. I still use it for some things. DAT tape is economical, but somewhat unreliable. DLT and LTO are very nice and seem to be reliable, but quite expensive. Many people are now buying large extra disk drives just to contain backups. That works well for short term failure recovery, but is less convenient for archival (very long term) backups. Disks are also less useful for extra large file systems - ones that are larger than the largest disk you can put on the machine. It is easier to to multiple media units on tape. Optical media works OK if the amount of data you will back up is small. The capacity of CD and DVD is small compared to modern disk and tape. Probably for disaster recovery, I would be inclined to suggest backups to disk if your backup requirements are withing the capacity of a drive you can make work on the system you wish to back up. So, once you have decided and acquired the backup media, then you have several choices for software. The most complete and general solution is to use dump(8) to make the backups. It handles all situations and types of files, directories, links and file ownership/permissions correctly. Nothing else does it 100%.You can easily write to either tape or disk with dump. Dump files are read by restore(8). The only weakness of dump is that it only works on whole file systems. That is great for most backups. But, if you want to back up just a few files or directories within a file system, then you might want to use tar(1) (with the -p flag to preserve as much of permissions as possible). Another thing to consider is to design your partitions such that the things you want to back up are put in a particular partition (file system) and things you will never want to back up are in a different partition. Then dump will get just what you want. That is one reason (not the only) for making /tmp a separate partition, for example. You don't generally bother to back up /tmp. Also, you may not want to back up '/' and /usr more than once or twice after an upgrade if you put users home directories elsewhere and move /usr/local to a different, frequently backed up partition. With just a little thought to layout, the whole file system limitation becomes moot and dump is _the_ choice. If you write dumps to a big disk, note that you should make a single large file system on it and write the dumps to files that include the dump date - something like: dump 0af /dumpdisk/root20061212 / presuming you have mounted that big disk file system as '/dumpdisk' and that you are dumping root on December 12, 2006. You will want to make a way of deleting obsolete dumps as well. jerry Thanking you Arun ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Backing up FREEBSD
Hello, If you have a spare machine with the diskspace neeeded then you might want to consider rsync over ssh. Rsync can do incremental backups, which can be nice and timesaving. There's plenty to read about it if you Google for Freebsd backing up rsync ssh ports. Tony Shadwick skrev: As with what he said. :) Dump and Restore are your friends. Also, in a crunch: tar -z -c -f /path/to/your/backup.tar.gz /filesystem Do that once for each mounted filesystem, and make sure backup.tar.gz lives on another system, perhaps on an nfs mount. Chad Gross wrote: The handbook is your friend: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/backup-basics.html and most likely: man dump Chad On 12/12/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Hi This is Arun from Singapore. I basically want to know how to back up files if a computer is already running on FREEBSD. Please help me with this as it is urgent. Thanking you Arun ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Backing up FREEBSD
* Roger Olofsson [EMAIL PROTECTED] [2006-12-12 18:23:50 +0100]: If you have a spare machine with the diskspace neeeded then you might want to consider rsync over ssh. Rsync can do incremental backups, which can be nice and timesaving. If you are going to go the rsync route, I recommend you check out rsnapshot, it's in /usr/ports/sysutils/rsnapshot, and their site is here: http://www.rsnapshot.org/ Thomas -- N.J. Thomas [EMAIL PROTECTED] Etiamsi occiderit me, in ipso sperabo ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
