Re: Looking for a Good FreeBSD and General Unix Backup System
I second BackupPC. Very nice, despite what some may consider a misleading name. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
After all the good suggestions by several people from this list, I think we will go with dump and restore. I used to use dump around ten years ago when our departmental Unix work station was a Sun and I don't ever remember it letting us down, even after something dreadful happened to the main drive and I had to rebuild it. There turns out to be a dump package for Linux which should give us the coverage we were looking for. The scripts I set up for doing backups are much smaller and easier to maintain with dump with the idea being that a shell script determines whether this is week 1 through 5 in a given month. The first day of that week is Level 0 followed by 6 more levels until we start another week such that the path to a backup looks like 02-1/systemname/systemname_0 systemname_1 and so forth. The defaults appear to be slightly different between the bsd dump and Linux's dump, but it wasn't difficult to get both to dump one volume containing all the inodes to a file which is actually a named pipe on the backup server. Many thanks to all. Martin McCormick ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
Mamlookie wrote: I just stumbled upon BackupPC yesterday, so I amnot sure how good it can be because I haven't had time to test, but nothing stops you from looking at it, now that you are after a solution. Please see http://backuppc.sourceforge.net PS: If you do test it out, please come back and tell us what you feel about it. I personally will appreciate the feedback, even if to my personal address. Backuppc is harder to configure, but allowed easier access to file version, even via web interface - can be useful if you want to give access to the backups to your customers. I personally use rsnapshot. Simply rocks :) Peter Zyumbilev ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
Mamlookie wrote, On 1/28/2009 1:29 AM: I just stumbled upon BackupPC yesterday, so I amnot sure how good it can be because I haven't had time to test, but nothing stops you from looking at it, now that you are after a solution. Please see http://backuppc.sourceforge.net PS: If you do test it out, please come back and tell us what you feel about it. I personally will appreciate the feedback, even if to my personal address. I use BackupPC to backup many machines at our school and a few remote sites I admin during the 'off hours'... All FreeBSD and a few Linux servers, all over sshd/rsync; it can also pull data from win32 machines as well, but I don't do that. I have ours setup with a backuppc 'server' running from thttpd on port 2359. Keeps all the apache non-sense from messing up the install.. (imho) and gives a platform independent answer if you run it on something else.. and doesn't mess up any current webserver you may have installed. It needs perl and a few modules (all of which are in ports) and runs with very minimal intervention once its done. Highly configurable, sends emails when there are problems, has many different ways to connect to remote machines.. etc.. if you are interested in hearing more about it let me know.. General Server Information * The servers PID is 36529, on host storage.phs.pcsd, version 3.1.0, started at 1/15 14:34. * This status was generated at 1/28 08:41. * The configuration was last loaded at 1/25 13:00. * PCs will be next queued at 1/28 09:00. * Other info: * 0 pending backup requests from last scheduled wakeup, * 0 pending user backup requests, * 0 pending command requests, * Pool is 102.00GB comprising 1152712 files and 4369 directories (as of 1/28 01:33), * Pool hashing gives 385 repeated files with longest chain 34, * Nightly cleanup removed 4700 files of size 0.05GB (around 1/28 01:33), * Pool file system was recently at 37% (1/28 08:32), today's max is 37% (1/28 01:00) and yesterday's max was 37%. /dev/mirror/gm0s1h330G113G190G37%/exports This is backing up about 9/10 servers atm. does incrementals once a day, and fulls once a week. Keeps the last 10 fulls, and at least 6 incrementals.. (all my settings) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
If you're looking for something serious (I mean with incremental/differential/full backups, retension periods, pools, multi platform, tape/file/dvd support, ...) I highly suggest Bacula (http://www.bacula.org). I use it successfully at work since two years (we used Amanda before) to backup 10+ machines (FreeBSD and Linux mainly) best regards, Julien On Tue, 2009-01-27 at 16:30 -0600, Martin McCormick wrote: Several months ago, I started using dar to backup a number of FreeBSD and Linux systems to one FreeBSD box. It worked fine once one got the syntax of the remote commands working, but then it all died when I moved it to a new FreeBSD6.3 system. If I can't figure out what is wrong or whether it is worth fixing, I am going to have to find some other archiver so we can get good backups and trust them to be easily restored. What we plan to do is backup a bunch of Unix systems to one FreeBSD box and then use a commercial package to back that box up to an enterprise-wide system we use. The archiver we need must be able to make 1 full backup of each system like tar and then incrementals until we are ready for another full backup. Any suggestions as to what is best? Dar seemed to be okay until the incrementals would hang each time with some error messages about the format version being too high which is bogus because we are using the same version for all the effected systems. The archive files should use tar or some other common storage method so we could unpack an archive from a Linux system in to a FreeBSD directory or vice versa. Any backup packages using tar would be fine as long as they can do incremental backups and use ssh as the transport. Any ideas are appreciated. Thank you. Martin McCormick WB5AGZ Stillwater, OK Systems Engineer OSU Information Technology Department Telecommunications Services Group ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org -- Julien Cigar Belgian Biodiversity Platform http://www.biodiversity.be Université Libre de Bruxelles (ULB) Campus de la Plaine CP 257 Bâtiment NO, Bureau 4 N4 115C (Niveau 4) Boulevard du Triomphe, entrée ULB 2 B-1050 Bruxelles Mail: jci...@ulb.ac.be @biobel: http://biobel.biodiversity.be/person/show/471 Tel : 02 650 57 52 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
If you're looking for something serious (I mean with incremental/differential/full backups, retension periods, pools, multi platform, tape/file/dvd support, ...) I highly suggest Bacula (http://www.bacula.org). how you define serious? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
On Wed 28 Jan 2009 at 10:38:56 PST Wojciech Puchar wrote: If you're looking for something serious (I mean with incremental/differential/full backups, retension periods, pools, multi platform, tape/file/dvd support, ...) I highly suggest Bacula (http://www.bacula.org). how you define serious? Um, isn't his definition already there, between the parentheses? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
On Tue, Jan 27, 2009 at 04:30:54PM -0600, Martin McCormick wrote: What we plan to do is backup a bunch of Unix systems to one FreeBSD box and then use a commercial package to back that box up to an enterprise-wide system we use. The archiver we need must be able to make 1 full backup of each system like tar and then incrementals until we are ready for another full backup. The best choice is dump, which was designed and built for exactly this purpose. (tar is fine for archives of static hierarchies, but it is not suitable for full-system backups.) Dump fully supports the concept of full/partial backups in a robust manner. (It has other useful features as well, notably its ability to deal with non-quiescent filesystems in a sensible way.) Excellent backup systems can be built with judicious use of dump -- you don't need to waste money on commercial products. [1] Depending on your specific requirements, it may be desirable to combine dump with other programs you already have (e.g., rysnc, gzip/bzip, scp, and so on). For example, I recently had occasion to build a system which backed up and replicated a multi-terabyte repository across a WAN. Using just the tools already on the system, and about 300 lines of shell (2/3 of which are comments), it wasn't that difficult to meet both requirements and do so in a way that minimized the bandwidth needed. This is really no big deal: it's just a matter of selecting the right tools and combining them -- which is the essence of the Unix philosophy. ---Rsk [1] Every commercial backup system I've evaluated for Unix -- over many, many years -- has produced inferior results. It pains me to watch people waste money on over-priced, under-performing and often-insecure commercial packages when they already have all the software they need...and just need to learn how to use it. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
purpose. (tar is fine for archives of static hierarchies, but it is not suitable for full-system backups.) Dump fully supports the concept of full/partial backups in a robust manner. (It has other useful features dump is perfect. period. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
On Wed, Jan 28, 2009 at 6:51 PM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: dump is perfect. period. Is it possible to pull out individual files? A fellow sysadmin asked me that years ago and I didn't have an answer for him. Thanks, Jaime ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
On Jan 28, 2009, at 16:52, Jaime wrote: On Wed, Jan 28, 2009 at 6:51 PM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: dump is perfect. period. Is it possible to pull out individual files? A fellow sysadmin asked me that years ago and I didn't have an answer for him. Most certainly. Use the restore function. Interactive mode is easiest for a small number of files. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
On Wed, Jan 28, 2009 at 07:52:40PM -0500, Jaime wrote: On Wed, Jan 28, 2009 at 6:51 PM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: dump is perfect. period. Is it possible to pull out individual files? A fellow sysadmin asked me that years ago and I didn't have an answer for him. Very easily.Just use restore -i Usually when I do that, I make a special direcory to receive things. (I usually call it 'unroll') That way I can put what I want there and then move it to where I want to, even if it is different from where it originally was. jerry Thanks, Jaime ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
On Wed, Jan 28, 2009 at 05:11:22PM -0800, Doug Hardie wrote: Most certainly. Use the restore function. Interactive mode is easiest for a small number of files. Doug's correct. The interactive mode of restore, with its shell-like interface, is probably easiest if you're just looking for two files or one directory or similar. The non-interactive mode, which has a syntax similar to tar's, is probably easiest if you're going to restore many files/directories, or if you're going to restore an entire filesystem. Two notes to tuck away for future use: First, when you use interactive mode, you use shell-like commands (e.g., cd, ls) to navigate the directory hierarchy and pick out what you want to restore. You add each one to a list (that restore keeps track of for you) and then, when you've selected them all, you tell restore to extract them. This is point where restore will tell you that you haven't read any tapes yet, and ask you what tape to read. Tell it 1. There's a long explanation behind this that has to do with the days when 1600 BPI 9-track tapes were backup media, and dumps often spanned multiple tapes, and so on. Second, restore runs in user mode, so when it restores a file (or all the files in an entire filesystem) it creates them through the same mechanism any other user-mode program would. That means, from Unix's point of view, they're new files: new inode number, and all that. So if you're doing a major restore -- say, an entire filesystem -- then you probably want to follow that up with a level 0 dump if you plan to do partial dumps. Otherwise, those partial dumps aren't going to have what you probably want them to have. Arguably this is inconvenient but (a) it's a rare circumstance (b) it's not THAT inconvenient and (c) there's no good way around it without sacrificing a lot of the power of dump. Okay, three notes: it's often advisable to create a scratch directory and restore into that, just in case you fumble-finger something. Given that you're restoring, which means something has gone wrong, possibly a big something, you may be stressed and hurried, and thinking that this would be the worst possible time for something ELSE to go wrong. A scratch directory insulates you from most of that. (No, of course this is entirely based on other peoples' experiences, it would never relate to my own...why do you ask?) ---Rsk ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
restore -i On Wed, 28 Jan 2009, Jaime wrote: On Wed, Jan 28, 2009 at 6:51 PM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: dump is perfect. period. Is it possible to pull out individual files? A fellow sysadmin asked me that years ago and I didn't have an answer for him. Thanks, Jaime ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Looking for a Good FreeBSD and General Unix Backup System
Several months ago, I started using dar to backup a number of FreeBSD and Linux systems to one FreeBSD box. It worked fine once one got the syntax of the remote commands working, but then it all died when I moved it to a new FreeBSD6.3 system. If I can't figure out what is wrong or whether it is worth fixing, I am going to have to find some other archiver so we can get good backups and trust them to be easily restored. What we plan to do is backup a bunch of Unix systems to one FreeBSD box and then use a commercial package to back that box up to an enterprise-wide system we use. The archiver we need must be able to make 1 full backup of each system like tar and then incrementals until we are ready for another full backup. Any suggestions as to what is best? Dar seemed to be okay until the incrementals would hang each time with some error messages about the format version being too high which is bogus because we are using the same version for all the effected systems. The archive files should use tar or some other common storage method so we could unpack an archive from a Linux system in to a FreeBSD directory or vice versa. Any backup packages using tar would be fine as long as they can do incremental backups and use ssh as the transport. Any ideas are appreciated. Thank you. Martin McCormick WB5AGZ Stillwater, OK Systems Engineer OSU Information Technology Department Telecommunications Services Group ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
Several months ago, I started using dar to backup a number of FreeBSD and Linux systems to one FreeBSD box. It worked fine once one got the syntax of the remote commands working, but then it all died when I moved it to a new FreeBSD6.3 system. if you are backup up to disk on same/other server, use rsync. cp -lpR is very useful for doing snapshots ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
On Tue, Jan 27, 2009 at 04:30:54PM -0600, Martin McCormick wrote: Several months ago, I started using dar to backup a number of FreeBSD and Linux systems to one FreeBSD box. It worked fine once one got the syntax of the remote commands working, but then it all died when I moved it to a new FreeBSD6.3 system. If I can't figure out what is wrong or whether it is worth fixing, I am going to have to find some other archiver so we can get good backups and trust them to be easily restored. Depends on _what you want to back up. For backing up FreeBSD system files, use dump(8). My machines have separate partitions for /, /usr and /var. These are all backed-up by making dumps. For backing up user data, like /home, I prefer rsync. If my machine breaks, I can restore the system from dumps using nothing but a boot CD and the dumps. (to get / and all ports back up and running), and then use rsync to restore the user data. What we plan to do is backup a bunch of Unix systems to one FreeBSD box and then use a commercial package to back that box up to an enterprise-wide system we use. The archiver we need must be able to make 1 full backup of each system like tar and then incrementals until we are ready for another full backup. Any suggestions as to what is best? Use the OS's native dump(8) command to back up the system partitions. It's the best way to do a bare metal recovery. For the rest, use rsync(1). Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpK3UDMRzGos.pgp Description: PGP signature
Re: Looking for a Good FreeBSD and General Unix Backup System
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin McCormick wrote: Several months ago, I started using dar to backup a number of FreeBSD and Linux systems to one FreeBSD box. It worked fine once one got the syntax of the remote commands working, but then it all died when I moved it to a new FreeBSD6.3 system. If I can't figure out what is wrong or whether it is worth fixing, I am going to have to find some other archiver so we can get good backups and trust them to be easily restored. What we plan to do is backup a bunch of Unix systems to one FreeBSD box and then use a commercial package to back that box up to an enterprise-wide system we use. The archiver we need must be able to make 1 full backup of each system like tar and then incrementals until we are ready for another full backup. Any suggestions as to what is best? Dar seemed to be okay until the incrementals would hang each time with some error messages about the format version being too high which is bogus because we are using the same version for all the effected systems. The archive files should use tar or some other common storage method so we could unpack an archive from a Linux system in to a FreeBSD directory or vice versa. Any backup packages using tar would be fine as long as they can do incremental backups and use ssh as the transport. Any ideas are appreciated. Thank you. Martin McCormick WB5AGZ Stillwater, OK Systems Engineer OSU Information Technology Department Telecommunications Services Group Hi Martin, It doesn't use tar, but I've had great luck with rsnapshot (http://www.freshports.org/sysutils/rsnapshot/ - http://www.rsnapshot.org/) for backing up multiple RedHat Linux and FreeBSD servers to a central server over SSH. rsnapshot is very flexible and you can set up your desired retention policies - hourly, daily, weekly and monthly. The destination backup tree is kept as small as possible with the use of hard links. Best of luck, Greg Larkin - -- Greg Larkin http://www.FreeBSD.org/ - The Power To Serve http://www.sourcehosting.net/ - Ready. Set. Code. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJf5ao0sRouByUApARAlhYAKCE6S4TKAo0NrQBfqpjgv4lbEt5nwCfadBf no05yvVwX5PId3H5MMjkKCg= =TTsE -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
Greg Larkin writes: -BEGIN PGP SIGNED MESSAGE- It doesn't use tar, but I've had great luck with rsnapshot (http://www.freshports.org/sysutils/rsnapshot/ - http://www.rsnapshot.org/) for backing up multiple RedHat Linux and FreeBSD servers to a central server over SSH. Thanks to each of you. I have actually experimented once with rsync and I appreciate knowing about rsnapshot. I think one of these plus dump may serve us well. Again, thanks for your help. Martin McCormick ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
Martin McCormick wrote: Several months ago, I started using dar to backup a number of FreeBSD and Linux systems to one FreeBSD box. It worked fine once one got the syntax of the remote commands working, but then it all died when I moved it to a new FreeBSD6.3 system. I feel for you. What we plan to do is backup a bunch of Unix systems to one FreeBSD box and then use a commercial package to back that box up to an enterprise-wide system we use. The archiver we need must be able to make 1 full backup of each system like tar and then incrementals until we are ready for another full backup. I am an AMANDA advocate. You seem to have a decent understanding of the difference between 'backup' and 'archive'. rsync does not fit your bill here, IMHO. Given that you need a 'standard' method of recovery, AMANDA conforms to dump(8) and restore(8) if you don't have easy/direct access to its internal amrecover(8) command set. The initial learning curve isn't bad for a sys admin who is familiar with performing proper network backups, and once initially configured, just does it's job. Since you can have all of your FreeBSD boxes backed up to a single hierarchical directory structure via AMANDA, your enterprise server should have no problem sweeping that single directory up, fulfilling that portion of the criteria. Any suggestions as to what is best? Dar seemed to be okay until the incrementals would hang each time with some error messages about the format version being too high which is bogus because we are using the same version for all the effected systems. My suggestion is to use something that conforms to age-old and tried-and-true dump/restore routines. A backup (as I can tell you already know) is as good as the time it takes to restore from it. The archive files should use tar or some other common storage method so we could unpack an archive from a Linux system in to a FreeBSD directory or vice versa. http://amanda.org Not only will it use tar, but you can define, on a partition level basis, which tar to use, whether to compress, etc etc. Any backup packages using tar would be fine as long as they can do incremental backups and use ssh as the transport. If you do use AMANDA, it is trivial to copy the backups over SSH whether it be after they are done or during backup. HTH. Steve ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Looking for a Good FreeBSD and General Unix Backup System
On Wed, Jan 28, 2009 at 1:30 AM, Martin McCormick mar...@dc.cis.okstate.edu wrote: Several months ago, I started using dar to backup a number of FreeBSD and Linux systems to one FreeBSD box. It worked fine once one got the syntax of the remote commands working, but then it all died when I moved it to a new FreeBSD6.3 system. If I can't figure out what is wrong or whether it is worth fixing, I am going to have to find some other archiver so we can get good backups and trust them to be easily restored. I just stumbled upon BackupPC yesterday, so I amnot sure how good it can be because I haven't had time to test, but nothing stops you from looking at it, now that you are after a solution. Please see http://backuppc.sourceforge.net PS: If you do test it out, please come back and tell us what you feel about it. I personally will appreciate the feedback, even if to my personal address. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ The only time a woman really succeeds in changing a man is when he is a baby. - Natalie Wood ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
