Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of m.r...@5-cent.us Sent: Friday, January 15, 2010 5:14 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server I was thinking about your long term here. Make sure to use LVM to create your underlaying partition. Then you can add disk space in the future without having to reformat everything and can just grow your ext3/ext4 partition instead. With six drives installed, there is no more space to add more drives in the chassis. But thanks for the hint! snip Ahh, but when you replace some of them with larger drives? Then I'll consider it. ;-) -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Les Mikesell Sent: Thursday, January 14, 2010 3:14 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server Yes, but if you use the epel rpm, either mount it at /var/lib/BackupPC or put a symlink there before the install. If you install from the sourceforge source there is an install script that modifies the location so you can put things where you want, but the rpm packages have already done that. The next version will make this easier to change but the current one needs to stay in the location set when the package was built. Ran into some problems and couldn't login to the web interface. The above helped, when tracking down the paths and symlinks. Thanks! So far, BackupPC looks good. Will start configuring it now and do some test backups later this afternoon. Darn users can't let me work in peace... ;-) -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: Today I have five 500GB-disks raided on linux machine. Remove one for parity and I have 2TB of real space available. Doing a 0+1, ie 1TB, would indeed be better as performance goes, but 1TB of space, well, it just isn't enough unfortunately. As it is now, the 2TB shebang is mounted as /backup. Does that count as a single filesystem? I was thinking about your long term here. Make sure to use LVM to create your underlaying partition. Then you can add disk space in the future without having to reformat everything and can just grow your ext3/ext4 partition instead. -- Benjamin Franz ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Benjamin Franz Sent: Friday, January 15, 2010 2:26 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server I was thinking about your long term here. Make sure to use LVM to create your underlaying partition. Then you can add disk space in the future without having to reformat everything and can just grow your ext3/ext4 partition instead. With six drives installed, there is no more space to add more drives in the chassis. But thanks for the hint! -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: So you need to be able to walk the fine line between these two. I'm trying. Something it just isn't enough. Although the boss has a soft spot for linux, as he also heads the CADD (Computer Aided Drug Design)-group. To put it into perspective, ask the manager how much it would cost the business if this data was unrecoverable? After that, if they still don't want to spend a few hundred $$s on the insurance, get it in writing that your manager understands the risk and print it out and post it on your office wall. Rather confrontative isn't it? Me being a Swede, I try to avoid those situations if possible, and find a compromise instead that both parties can live with. 8-} Oh, and I'm a government employee, so the money I spend is tax-payers money. Got to be careful there. Being careful with the money is the point. Someone has to understand the risks. You know how that saying goes? You can chose between good, fast and cheap. But you're only ever allowed to pick any two. For me that's IT in a nutshell. ;-) The other question to ask is whether an offsite copy is needed. After a fire or other site disaster some businesses might collect the insurance money and disappear - others might want to be able to rebuild and continue. Government operations would probably need to continue and need a plan for that. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Benjamin Franz Sent: Friday, January 15, 2010 2:26 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server I was thinking about your long term here. Make sure to use LVM to create your underlaying partition. Then you can add disk space in the future without having to reformat everything and can just grow your ext3/ext4 partition instead. With six drives installed, there is no more space to add more drives in the chassis. But thanks for the hint! Ok. Oh, one last thing. Don't forget to use the '-E stride=XX,stripe-width=YY (where XX and YY are replaced with the appropriate values) options creating your filesystem on the RAID. Otherwise your disk drive usage will have 'hot spots' and slower than optimal speed. Do a man mke2fs to understand how to use them correctly. -- Benjamin Franz ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: Yes, but if you use the epel rpm, either mount it at /var/lib/BackupPC or put a symlink there before the install. If you install from the sourceforge source there is an install script that modifies the location so you can put things where you want, but the rpm packages have already done that. The next version will make this easier to change but the current one needs to stay in the location set when the package was built. Ran into some problems and couldn't login to the web interface. The above helped, when tracking down the paths and symlinks. Thanks! So far, BackupPC looks good. Will start configuring it now and do some test backups later this afternoon. Darn users can't let me work in peace... ;-) You might want to join the mail lists: http://backuppc.sourceforge.net/info.html#lists if you have any specific questions about it. There are several users with a lot of experience and the author still participates. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Les Mikesell Sent: Friday, January 15, 2010 2:42 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server The other question to ask is whether an offsite copy is needed. After a fire or other site disaster some businesses might collect the insurance money and disappear - others might want to be able to rebuild and continue. Government operations would probably need to continue and need a plan for that. Nah, no need for that. We have the client machines in one end of the house and server room, where the backup server will reside eventually, in another. They are like five stories and about 200m as the bird flies, apart. I think we're covered enough. Had some initial problems with the four bytes read-error, but some more reading about ssh-rsa in the howto seem to have solved the problem. -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Benjamin Franz Sent: Friday, January 15, 2010 2:43 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server With six drives installed, there is no more space to add more drives in the chassis. But thanks for the hint! Ok. Oh, one last thing. Don't forget to use the '-E stride=XX,stripe-width=YY (where XX and YY are replaced with the appropriate values) options creating your filesystem on the RAID. Otherwise your disk drive usage will have 'hot spots' and slower than optimal speed. Do a man mke2fs to understand how to use them correctly. That sounds familiar. I think I read about that when I created the array, but opted for an automatic setup anyway. I'll check up on that again. Thanks! -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Les Mikesell Sent: Friday, January 15, 2010 2:49 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server You might want to join the mail lists: http://backuppc.sourceforge.net/info.html#lists if you have any specific questions about it. There are several users with a lot of experience and the author still participates. Done. Thanks! -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
From: Benjamin Franz jfr...@freerun.com Oh, one last thing. Don't forget to use the '-E stride=XX,stripe-width=YY (where XX and YY are replaced with the appropriate values) options creating your filesystem on the RAID. Otherwise your disk drive usage will have 'hot spots' and slower than optimal speed. Do a man mke2fs to understand how to use them correctly. I think 'stripe-width' is sadly no more available... isn't it? JD ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Benjamin Franz Sent: Friday, January 15, 2010 2:26 PM I was thinking about your long term here. Make sure to use LVM to create your underlaying partition. Then you can add disk space in the future without having to reformat everything and can just grow your ext3/ext4 partition instead. With six drives installed, there is no more space to add more drives in the chassis. But thanks for the hint! snip Ahh, but when you replace some of them with larger drives? mark ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On Wed, Jan 13, 2010 at 5:20 PM, Jussi Hirvi listmem...@greenspot.fi wrote: I think nobody has yet mentioned rdiff-backup. I have very good experiences with it. Easy to setup and control (only remember first to install the required packages, and I think rsync-devel was not mentioned but is required). Rdiff-backup keeps up an exact copy of the source director(ies), plus it maintains a separate directory for deleted/changed items. With an appropriate command you can restore the source directory as it was at a given point of time. Very neat, and space-saving. http://rdiff-backup.nongnu.org/ - Jussi yes, i can agree. also using rdiff-backup and can say that it is very good sollution. even tested it by restoring few terabaytes of information. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Max Hetrick Sent: Wednesday, January 13, 2010 4:39 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server The reason I really like BackupPC is the compression you can get. It really helps me out since I have a smaller dedicated backup server. For my instance, since I have a dedicated server, I'd trade performance for the space compression saves me. I'll look into Backuppc as the primary solution for our linux-group. Just need to find a space to put this dual-xeon, it's very loud, as I need it handy while trying this out. Thanks for the info. -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Les Mikesell Sent: Wednesday, January 13, 2010 4:57 PM To: centos@centos.org Subject: Re: [CentOS] Backup server On 1/13/2010 9:08 AM, Gabriel Rosca wrote: My google searches would have me believe that Amanda is the more popular choice for backup on linux. On this list it seems Backuppc is. Strange... ;-) Amanda is good for tape, and has a nice feature of being able to estimate the sizes of full and incremental runs ahead of time and make adjustments to make them all fit on the available tape. It can save to disk but doesn't do any pooling. Backuppc is mostly designed for online backups. It can archive tar images out to tape but it's an afterthought and not great at it. Both are pretty much 'set up and forget' programs although with amanda you do have to swap the tape every day. Aha! Online backups is what we use otherwise for Windows here. Seems like Backuppc is really the way to here. Thanks! FWIW, we used tapes a handful of years back but it was just too much data to transfer, and there was no budget to get an eight-tape-robot, so we opted for an online homebrew hd-based solution. That has worked ever since. I'm very happy with that. -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Les Mikesell Sent: Wednesday, January 13, 2010 5:17 PM To: centos@centos.org Subject: Re: [CentOS] Backup server I've never had any problems with software raid5 in linux before, but you never know... There's a big write performance hit from raid5 (software or not). It may not be enough to be a showstopper but I wouldn't recommend it. Can you reconfigure to a 0+1 or some other type that has better performance without losing too much space? The archive does have to be on a single filesystem, though, and if you use the epel RPM it makes things easier if you mount the volume at /var/lib/BackupPC before the install. See 'how to change archive directory' from: http://sourceforge.net/apps/mediawiki/backuppc/index.php?title=Tips_and_Tricks to do it after the install. I know, but in linux it's still a lot better than Windows, which I today regret I ever introduced it in... Don't get me wrong, reliability is fine, but the recheck on every restart is kinda' bothersome and takes like forever Today I have five 500GB-disks raided on linux machine. Remove one for parity and I have 2TB of real space available. Doing a 0+1, ie 1TB, would indeed be better as performance goes, but 1TB of space, well, it just isn't enough unfortunately. As it is now, the 2TB shebang is mounted as /backup. Does that count as a single filesystem? -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of S.Tindall Sent: Wednesday, January 13, 2010 8:30 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server So just use the stock epel package and you don't need to modify apache. Thanks! -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
From: Sorin Srbu sorin.s...@orgfarm.uu.se My google searches would have me believe that Amanda is the more popular choice for backup on linux. On this list it seems Backuppc is. Strange... amanda was created in 1991... BackupPC in 2001. That would explain it a bit... Also if tapes are the backup media, which was mainly the case a 'few' years ago (still?), Backuppc won't be an option afaik... JD ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: Today I have five 500GB-disks raided on linux machine. Remove one for parity and I have 2TB of real space available. Doing a 0+1, ie 1TB, would indeed be better as performance goes, but 1TB of space, well, it just isn't enough unfortunately. As it is now, the 2TB shebang is mounted as /backup. Does that count as a single filesystem? If you have any budget at all, invest in bigger drives. 7200 RPM 1 TB RAID rated drives can be bought for $160 each. Desktop rated 5900 RPM 1.5 TB drives (which you can probably get away with in a dedicated backup server since you don't care a lot about speed and can tolerate long pauses for sector repair) can be bought for $110 each. Check Newegg. Second, to maximize 'depth' of backups you should use a 'Tower of Hanoi'-like backup system. For example: 1 day 2 days 4 days 8 days 16 days etc If your selected backup software supports either hardlinking or plain old incremental backups that will keep the size of backups down a lot while giving you history. -- Benjamin Franz ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: I've never had any problems with software raid5 in linux before, but you never know... There's a big write performance hit from raid5 (software or not). It may not be enough to be a showstopper but I wouldn't recommend it. Can you reconfigure to a 0+1 or some other type that has better performance without losing too much space? The archive does have to be on a single filesystem, though, and if you use the epel RPM it makes things easier if you mount the volume at /var/lib/BackupPC before the install. See 'how to change archive directory' from: http://sourceforge.net/apps/mediawiki/backuppc/index.php?title=Tips_and_Tricks to do it after the install. I know, but in linux it's still a lot better than Windows, which I today regret I ever introduced it in... Don't get me wrong, reliability is fine, but the recheck on every restart is kinda' bothersome and takes like forever Today I have five 500GB-disks raided on linux machine. Remove one for parity and I have 2TB of real space available. Doing a 0+1, ie 1TB, would indeed be better as performance goes, but 1TB of space, well, it just isn't enough unfortunately. If you have any opportunity to change things, I'd get some larger drives and use raid1 or 0+1. If you want offsite copies, a workable approach in the 2TB scale is to make a 3-member raid1 where you periodically swap the 3rd drive (in an internal or external swappable sata enclosure). If you don't need that, a 4 drive 0+1 raid of 1.5 TB drives would give you 3TB and better performance. What you have will work - but it will limit performance. As it is now, the 2TB shebang is mounted as /backup. Does that count as a single filesystem? Yes, but if you use the epel rpm, either mount it at /var/lib/BackupPC or put a symlink there before the install. If you install from the sourceforge source there is an install script that modifies the location so you can put things where you want, but the rpm packages have already done that. The next version will make this easier to change but the current one needs to stay in the location set when the package was built. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Benjamin Franz Sent: Thursday, January 14, 2010 2:12 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server If you have any budget at all, invest in bigger drives. 7200 RPM 1 TB RAID rated drives can be bought for $160 each. Desktop rated 5900 RPM 1.5 TB drives (which you can probably get away with in a dedicated backup server since you don't care a lot about speed and can tolerate long pauses for sector repair) can be bought for $110 each. Check Newegg. I haven't got a budget really. Today I asked for a new group-printer today and the boss looked pained... 8-} I opted for the proven 500GB-sized disks and got more of those instead. I've had a handful of 750GB-drives die on me recently. Somehow it feels the technology isn't quite there yet for the bigger drive-sizes. Anybody remember the IBM Deskstars in the early 00's...? Also, my experience is the more smaller disks you have, the faster they get. Less to write to each I guess. Second, to maximize 'depth' of backups you should use a 'Tower of Hanoi'-like backup system. Good advice, thanks! -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Les Mikesell Sent: Thursday, January 14, 2010 3:14 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server If you have any opportunity to change things, I'd get some larger drives and use raid1 or 0+1. If you want offsite copies, a workable approach in the 2TB scale is to make a 3-member raid1 where you periodically swap the 3rd drive (in an internal or external swappable sata enclosure). If you don't need that, a 4 drive 0+1 raid of 1.5 TB drives would give you 3TB and better performance. What you have will work - but it will limit performance. Nice idea, but no budget. Sorry. Maybe next year. This year is for getting this thing started at all, and get the backups going. Therefore I opted for most possible space with some redundancy. Not the best solution, but workable. As it is now, the 2TB shebang is mounted as /backup. Does that count as a single filesystem? Yes, but if you use the epel rpm, either mount it at /var/lib/BackupPC or put a symlink there before the install. If you install from the sourceforge source there is an install script that modifies the location so you can put things where you want, but the rpm packages have already done that. The next version will make this easier to change but the current one needs to stay in the location set when the package was built. It'll probably be epel. Symlink's probably the easiest way to do it. Thanks for the hint. smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Benjamin Franz Sent: Thursday, January 14, 2010 2:12 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server If you have any budget at all, invest in bigger drives. 7200 RPM 1 TB RAID rated drives can be bought for $160 each. Desktop rated 5900 RPM 1.5 TB drives (which you can probably get away with in a dedicated backup server since you don't care a lot about speed and can tolerate long pauses for sector repair) can be bought for $110 each. Check Newegg. I haven't got a budget really. Today I asked for a new group-printer today and the boss looked pained... 8-} SATA disks fit into 'office supply' budgets. I opted for the proven 500GB-sized disks and got more of those instead. I've had a handful of 750GB-drives die on me recently. Go to the vendor's web site, enter their serial numbers and get an RMA for a free replacement. Every vendor has had bad batches. Somehow it feels the technology isn't quite there yet for the bigger drive-sizes. Anybody remember the IBM Deskstars in the early 00's...? They replace them too, within the warranty period. This is the reason you are making backups, remember. Things break. Also, my experience is the more smaller disks you have, the faster they get. Less to write to each I guess. That's true when the heads seek independently. With raid5 you lock onto the slowest of the set unless you have a very large number of drives. Second, to maximize 'depth' of backups you should use a 'Tower of Hanoi'-like backup system. Good advice, thanks! Backuppc will take care of that for you. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Les Mikesell Sent: Thursday, January 14, 2010 3:55 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server I haven't got a budget really. Today I asked for a new group-printer today and the boss looked pained... 8-} SATA disks fit into 'office supply' budgets. That's exactly what I bought actually. I opted for the proven 500GB-sized disks and got more of those instead. I've had a handful of 750GB-drives die on me recently. Go to the vendor's web site, enter their serial numbers and get an RMA for a free replacement. Every vendor has had bad batches. Already done. Still feel a bit burned by the whole matter though. -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On Thu, Jan 14, 2010 at 9:37 AM, Sorin Srbu sorin.s...@orgfarm.uu.se wrote: -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Benjamin Franz Sent: Thursday, January 14, 2010 2:12 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server If you have any budget at all, invest in bigger drives. 7200 RPM 1 TB RAID rated drives can be bought for $160 each. Desktop rated 5900 RPM 1.5 TB drives (which you can probably get away with in a dedicated backup server since you don't care a lot about speed and can tolerate long pauses for sector repair) can be bought for $110 each. Check Newegg. I haven't got a budget really. Today I asked for a new group-printer today and the boss looked pained... 8-} I opted for the proven 500GB-sized disks and got more of those instead. I've had a handful of 750GB-drives die on me recently. Somehow it feels the technology isn't quite there yet for the bigger drive-sizes. Anybody remember the IBM Deskstars in the early 00's...? Also, my experience is the more smaller disks you have, the faster they get. Less to write to each I guess. Second, to maximize 'depth' of backups you should use a 'Tower of Hanoi'-like backup system. Good advice, thanks! -- /Sorin There seems to be a persistent conception among managers that anything IT related is a huge capital expenditure (as it used to be), and there's all sorts of resistance to buying anything new. However, you probably spend more on printer paper in 1 week than it costs to buy a 1TB drive. This kind of equipment is a disposable commodity, even though the accounting department still prefers to write it off over 7 years. However, IT also has a reputation of always wanting to buy new toys. Many times these toys are not needed, even though the IT person insists that they are. So you need to be able to walk the fine line between these two. To put it into perspective, ask the manager how much it would cost the business if this data was unrecoverable? After that, if they still don't want to spend a few hundred $$s on the insurance, get it in writing that your manager understands the risk and print it out and post it on your office wall. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Brian Mathis wrote: To put it into perspective, ask the manager how much it would cost the business if this data was unrecoverable? After that, if they still don't want to spend a few hundred $$s on the insurance, get it in writing that your manager understands the risk and print it out and post it on your office wall. I was just getting ready to say this. Ask how much it will cost them when they need to pull something from a backup, that they've accidentally deleted and need back. It really doesn't cost that much to build a small server. I built my own using a 3Ware drive cage and 4 SATA drives. I have 1TB of storage for my backup server. I think I only spent around $2,000 to build it. I'm starting to run out of space now, but we're looking at a cheaper iSCSI SAN to attach to this machine to expand on. At any rate, you really don't have to spend a lot of money to get something decent up and running. And even if you spend some, you need to explain to management that backups are extremely important. Once you get something in place then, it's important to actually test them and check on them that you're backing up. That's what led me to BackupPC in the first place. We used to use rsnapshot here, and there were quite a few customized hacked together things that we thought were running nightly, and they really weren't. So, when I started investigating, I realized that our backups here hadn't been taking place for over a month. Bad! So, I found BackuPC to replace rsnapshot, and have been happy since then for our online offsite backups. Regards, Max ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On 1/14/2010 9:08 AM, Sorin Srbu wrote: Go to the vendor's web site, enter their serial numbers and get an RMA for a free replacement. Every vendor has had bad batches. Already done. Still feel a bit burned by the whole matter though. Unfortunately, the only way commodity priced things get large scale real-world testing is after a large number of them have been sold. If the first one sold had to be perfect none of us could afford to buy it. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On 1/14/2010 9:23 AM, Max Hetrick wrote: That's what led me to BackupPC in the first place. We used to use rsnapshot here, and there were quite a few customized hacked together things that we thought were running nightly, and they really weren't. So, when I started investigating, I realized that our backups here hadn't been taking place for over a month. Bad! So, I found BackuPC to replace rsnapshot, and have been happy since then for our online offsite backups. Backuppc will at least send you an email when the backups have failed for 3 days in a row. I probably should mention the one scenario it doesn't handle very well, though. If you have very large files that have frequent small changes (active databases, logs, unix mailboxes, etc.), backuppc will store a complete new copy on every run, even though it may use rsync to only transfer the differences. You may gain some space from compression, but the pooling scheme only works for files that are completely identical. File systems like zfs with block-level deduplication might be the best solution to deal with cases like that, or maybe rdiff-backup. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: Already done. Still feel a bit burned by the whole matter though. I still feel burned from IBM's 75GXP fiasco ~7 years ago, even joined the lawsuit at the time(and got booted by the judge because I was in another state), had probably an 80% failure rate on those disks. I got a PDF on a CD somewhere that has all kinds of internal IBM docs(from the lawsuit) showing how they knew what the problems were but refused to fix them. OT but reminded me of that.. nate ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: Already done. Still feel a bit burned by the whole matter though. I still feel burned from IBM's 75GXP fiasco ~7 years ago, even joined the lawsuit at the time(and got booted by the judge because I was in another state), had probably an 80% failure rate on those disks. I got a PDF on a CD somewhere that has all kinds of internal IBM docs(from the lawsuit) showing how they knew what the problems were but refused to fix them. OT but reminded me of that.. Seagate Barracudas - mid-nineties, and again three-four years ago. Mid-nineties, first time as a sysadmin, and in nine months, *five* out of... was it eight? failed, one *twice*. The Sun account rep for who I worked for knew me by name I won't *ever* touch a Barracuda willingly. mark ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Les Mikesell wrote: Backuppc will at least send you an email when the backups have failed for 3 days in a row. Yeah, I have this configured. Although, to be honest since I've set it up, I've not had any failures yet, so I'll have to wait until I do, ha. Max ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On 1/14/2010 10:04 AM, m.r...@5-cent.us wrote: Already done. Still feel a bit burned by the whole matter though. I still feel burned from IBM's 75GXP fiasco ~7 years ago, even joined the lawsuit at the time(and got booted by the judge because I was in another state), had probably an 80% failure rate on those disks. I got a PDF on a CD somewhere that has all kinds of internal IBM docs(from the lawsuit) showing how they knew what the problems were but refused to fix them. OT but reminded me of that.. Seagate Barracudas - mid-nineties, and again three-four years ago. Mid-nineties, first time as a sysadmin, and in nine months, *five* out of... was it eight? failed, one *twice*. The Sun account rep for who I worked for knew me by name I won't *ever* touch a Barracuda willingly. That's not a particularly useful reaction because every vendor has shipped bad batches and it's a toss of the dice who will be next. Better to avoid short warranties and bad customer service - and never use the same model/batch for your backups as the live systems. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On 1/14/2010 10:04 AM, m.r...@5-cent.us wrote: Already done. Still feel a bit burned by the whole matter though. I still feel burned from IBM's 75GXP fiasco ~7 years ago, even snip OT but reminded me of that.. Seagate Barracudas - mid-nineties, and again three-four years ago. Mid-nineties, first time as a sysadmin, and in nine months, *five* out of... was it eight? failed, one *twice*. The Sun account rep for who I worked for knew me by name I won't *ever* touch a Barracuda willingly. That's not a particularly useful reaction because every vendor has shipped bad batches and it's a toss of the dice who will be next. Better to avoid short warranties and bad customer service - and never use the same model/batch for your backups as the live systems. Ah, no. Back in the eighties and early nineties, I thought highly of Seagates. Then, in the mid-nineties, *every* *single* ISP in Chicago had dumped the then-new Seagate Barracudas... and not a year after that, I got stuck with them in the external drives for my Sun, and the problem - note that I said one of them was replaced *twice* - clearly lasted for at least a couple of years. Then, about 4 years ago (plus or minus a year), I was hearing the same thing. It appears to me that Seagate, esp. with the Barracuda line, has a tendency to rush them out the door with clearly inadequate quality control. They've done it twice, ten years apart, so I take that as an institutional failing. mark probably too many MBA's ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On 1/14/2010 10:41 AM, m.r...@5-cent.us wrote: On 1/14/2010 10:04 AM, m.r...@5-cent.us wrote: Already done. Still feel a bit burned by the whole matter though. I still feel burned from IBM's 75GXP fiasco ~7 years ago, even snip OT but reminded me of that.. Seagate Barracudas - mid-nineties, and again three-four years ago. Mid-nineties, first time as a sysadmin, and in nine months, *five* out of... was it eight? failed, one *twice*. The Sun account rep for who I worked for knew me by name I won't *ever* touch a Barracuda willingly. That's not a particularly useful reaction because every vendor has shipped bad batches and it's a toss of the dice who will be next. Better to avoid short warranties and bad customer service - and never use the same model/batch for your backups as the live systems. Ah, no. Back in the eighties and early nineties, I thought highly of Seagates. Then, in the mid-nineties, *every* *single* ISP in Chicago had dumped the then-new Seagate Barracudas... and not a year after that, I got stuck with them in the external drives for my Sun, and the problem - note that I said one of them was replaced *twice* - clearly lasted for at least a couple of years. Then, about 4 years ago (plus or minus a year), I was hearing the same thing. It appears to me that Seagate, esp. with the Barracuda line, has a tendency to rush them out the door with clearly inadequate quality control. They've done it twice, ten years apart, so I take that as an institutional failing. But they weren't the only ones - just perhaps the biggest volume vendor, generally for good reasons. Name someone that hasn't shipped a bad drive - that we can afford. Besides, single drive failures should really be the least of your problems because they are common enough that ordinary OS's and controllers have simple techniques to deal with them. But there are a near-infinite number of other things that can go wrong that you also need to be prepared for. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On 1/14/2010 11:10 AM, Jussi Hirvi wrote: On 14.1.2010 17:59, Les Mikesell wrote: Backuppc will at least send you an email when the backups have failed for 3 days in a row. I probably should mention the one scenario it doesn't handle very well, though. If you have very large files that have frequent small changes (active databases, logs, unix mailboxes, etc.), backuppc will store a complete new copy on every run, Another scenario that Backuppc probably does not handle well is backups over the internet of workstations that are behind a firewall. I was hoping the remote users could start the backup of their own workstations using Backuppc:s web interface, but it does not seem possible (even though I haven't tried it in practice), because Backuppc uses netbios names to find the remote machine, and netbios names are not very well routable, I have read. In fact I don't know how this kind of backup service could be accomplished. The design expects the server to be able to be able to establish the connection to the targets so the straightforward approach would be to use a VPN like openvpn with a fixed private address when the tunnel is up. But it has been discussed on the mailing list and others have come up with ways to do it through ssh port-forwarding with the initial connection established from the remote side. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Brian Mathis Sent: Thursday, January 14, 2010 4:07 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server [...] So you need to be able to walk the fine line between these two. I'm trying. Something it just isn't enough. Although the boss has a soft spot for linux, as he also heads the CADD (Computer Aided Drug Design)-group. To put it into perspective, ask the manager how much it would cost the business if this data was unrecoverable? After that, if they still don't want to spend a few hundred $$s on the insurance, get it in writing that your manager understands the risk and print it out and post it on your office wall. Rather confrontative isn't it? Me being a Swede, I try to avoid those situations if possible, and find a compromise instead that both parties can live with. 8-} Oh, and I'm a government employee, so the money I spend is tax-payers money. Got to be careful there. You know how that saying goes? You can chose between good, fast and cheap. But you're only ever allowed to pick any two. For me that's IT in a nutshell. ;-) -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Les Mikesell Sent: Thursday, January 14, 2010 4:48 PM To: centos@centos.org Subject: Re: [CentOS] Backup server Unfortunately, the only way commodity priced things get large scale real-world testing is after a large number of them have been sold. If the first one sold had to be perfect none of us could afford to buy it. Yupp, but did it have to be me? ;-) -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of nate Sent: Thursday, January 14, 2010 5:00 PM To: centos@centos.org Subject: Re: [CentOS] Backup server I still feel burned from IBM's 75GXP fiasco ~7 years ago, even joined the lawsuit at the time(and got booted by the judge because I was in another state), had probably an 80% failure rate on those disks. I got a PDF on a CD somewhere that has all kinds of internal IBM docs(from the lawsuit) showing how they knew what the problems were but refused to fix them. Been there done that. We bought a dozen or so OEM-machines at the time, all of them using that particular drive. All harddrives died after a year or so IIRC, and all of them within a two-week period... It still bugs me. 8-/ -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] Backup server
Hi all, I've built a new backup server for our linux-clients. Is Amanda the way to go for a backup-solution? It seems to be pretty powerful, if a bit finickety to set up initially. The way we currently do backups is to use rsync from the clients to two folders on an older server that rolls over every other week. This worked fine for a while, but the rsync is cumulative and the users generate a tremendous amount of data every day after having had a client upgrade with newer and hilariously fast computers for calculation. The previous *nix-admin set it up this way with rsync, meaning that we in the long run have data that is way obsolete and get increasingly difficult to maintain. As the backup solution must be next to free, ie free beer, Amanda looks suitable. What do you use for backing up data? -- BW, Sorin --- # Sorin Srbu[Sysadmin, Systems Engineer] # Dept of Medicinal Chemistry, Phone: +46 (0)18-4714482 3 signals GSM # Div of Org Pharm Chem,Mobile: +46 (0)701-718023 # Box 574, Uppsala University, Fax: +46 (0)18-4714482 # SE-751 23 Uppsala, Sweden Visit: BMC, Husargatan 3, D5:512b # Web: http://www.orgfarm.uu.se --- # () ASCII ribbon campaign - Against html E-mail # /\ # # MotD follows: # One tentacle, one vote. smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
From: Sorin Srbu sorin.s...@orgfarm.uu.se I've built a new backup server for our linux-clients. Is Amanda the way to go for a backup-solution? It seems to be pretty powerful, if a bit finickety to set up initially. The way we currently do backups is to use rsync from the clients to two folders on an older server that rolls over every other week. This worked fine for a while, but the rsync is cumulative and the users generate a tremendous amount of data every day after having had a client upgrade with newer and hilariously fast computers for calculation. The previous *nix-admin set it up this way with rsync, meaning that we in the long run have data that is way obsolete and get increasingly difficult to maintain. As the backup solution must be next to free, ie free beer, Amanda looks suitable. What do you use for backing up data? Used amanda in the past (a decade ago)... I use bacula now. JD ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: Hi all, I've built a new backup server for our linux-clients. Is Amanda the way to go for a backup-solution? It seems to be pretty powerful, if a bit finickety to set up initially. The way we currently do backups is to use rsync from the clients to two folders on an older server that rolls over every other week. This worked fine for a while, but the rsync is cumulative and the users generate a tremendous amount of data every day after having had a client upgrade with newer and hilariously fast computers for calculation. The previous *nix-admin set it up this way with rsync, meaning that we in the long run have data that is way obsolete and get increasingly difficult to maintain. As the backup solution must be next to free, ie free beer, Amanda looks suitable. What do you use for backing up data? You can check out BackupPC as well. That's what I use. Regards, Max ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On Wed, Jan 13, 2010 at 5:04 AM, Sorin Srbu sorin.s...@orgfarm.uu.se wrote: snip The way we currently do backups is to use rsync from the clients to two folders on an older server that rolls over every other week. This worked fine for a while, but the rsync is cumulative and the users generate a tremendous amount of data... snip You might want to check out the rsync switches --backup-dir and --suffix. Using them some thing like this: --delete --backup --backup-dir=$MIRROR_DIR/RsyncBckups --suffix=.$DATE allows you to keep an exact duplication of the original directory and keeping the original files that were either deleted or overwritten in a seperate backup directory with dated suffixes, which can be archived on some regular basis. This should allow you to keep the simplicity of rsync and control the cumulative size. Brett ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On Wed, Jan 13, 2010 at 5:04 AM, Sorin Srbu sorin.s...@orgfarm.uu.se wrote: snip The way we currently do backups is to use rsync from the clients to two folders on an older server that rolls over every other week. This worked fine for a while, but the rsync is cumulative and the users generate a tremendous amount of data... I use rsnapshot .. which manages sets of rsync backups using hardlinks. IT may be similar to what you are using already .. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: Hi all, I've built a new backup server for our linux-clients. Is Amanda the way to go for a backup-solution? It seems to be pretty powerful, if a bit finickety to set up initially. The way we currently do backups is to use rsync from the clients to two folders on an older server that rolls over every other week. This worked fine for a while, but the rsync is cumulative and the users generate a tremendous amount of data every day after having had a client upgrade with newer and hilariously fast computers for calculation. The previous *nix-admin set it up this way with rsync, meaning that we in the long run have data that is way obsolete and get increasingly difficult to maintain. As the backup solution must be next to free, ie free beer, Amanda looks suitable. What do you use for backing up data? Backuppc is good for this - it can use rsync for the transfers (or tar or smb) but all duplicate data is compressed and pooled even if found on different machines and it has an easy setup to control how long old copies are retained. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
+1 for Backuppc. On Wed, Jan 13, 2010 at 7:30 AM, Les Mikesell lesmikes...@gmail.com wrote: Sorin Srbu wrote: Hi all, I've built a new backup server for our linux-clients. Is Amanda the way to go for a backup-solution? It seems to be pretty powerful, if a bit finickety to set up initially. The way we currently do backups is to use rsync from the clients to two folders on an older server that rolls over every other week. This worked fine for a while, but the rsync is cumulative and the users generate a tremendous amount of data every day after having had a client upgrade with newer and hilariously fast computers for calculation. The previous *nix-admin set it up this way with rsync, meaning that we in the long run have data that is way obsolete and get increasingly difficult to maintain. As the backup solution must be next to free, ie free beer, Amanda looks suitable. What do you use for backing up data? Backuppc is good for this - it can use rsync for the transfers (or tar or smb) but all duplicate data is compressed and pooled even if found on different machines and it has an easy setup to control how long old copies are retained. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Barry Brimer Sent: Wednesday, January 13, 2010 2:15 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server I use rsnapshot .. which manages sets of rsync backups using hardlinks. IT may be similar to what you are using already .. Somewhat similar, thanks. I think however I need to get away from this sort of backups. They're just to space-consuming. -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: Somewhat similar, thanks. I think however I need to get away from this sort of backups. They're just to space-consuming. Check out the user submitted HowTos on the Wiki. http://wiki.centos.org/HowTos#head-bab8e87dc82e722540e2d39de8408750004a8c4a Regards, Max ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Barry Brimer Sent: Wednesday, January 13, 2010 2:15 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server I use rsnapshot .. which manages sets of rsync backups using hardlinks. IT may be similar to what you are using already .. Somewhat similar, thanks. I think however I need to get away from this sort of backups. They're just to space-consuming. Between compression and pooling, I get about 10x the raw data being archived with backuppc - it beats juggling tapes and you can let the users access the backups of their own machine through a web interface. There are some down sides to plan around though: the compression takes some CPU and is slower than a stock rsync run, and the pooling is done with hardlinks which forces the archive to be on a single filesystem and makes it hard to duplicate for offsite copies. There's an RPM in epel that is easy to install on Centos. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
BackupPC over here - very happy with it for Linux and Windoze, at home and work -- “Don't eat anything you've ever seen advertised on TV” - Michael Pollan, author of In Defense of Food ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Alan McKay wrote: BackupPC over here - very happy with it for Linux and Windoze, at home and work I too have found BackupPC a marvelously simple-to-use program. In fact it seems to me much better for backing up Windows XP than Windows own Backup program, which I have never completely understood. Eg how exactly does one recover a lost file or folder with it? Whereas I have been saved several times in restoring lost material with BackupPC, including the contents of one entire damaged drive. -- Timothy Murphy e-mail: gayleard /at/ eircom.net tel: +353-86-2336090, +353-1-2842366 s-mail: School of Mathematics, Trinity College, Dublin 2, Ireland ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Max Hetrick Sent: Wednesday, January 13, 2010 2:36 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server Sorin Srbu wrote: Somewhat similar, thanks. I think however I need to get away from this sort of backups. They're just to space-consuming. Check out the user submitted HowTos on the Wiki. http://wiki.centos.org/HowTos#head- bab8e87dc82e722540e2d39de8408750004a8c4a It almost drives one to tears. I found some other Howtos, but this one is better. Thanks! -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Les Mikesell Sent: Wednesday, January 13, 2010 2:49 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server Between compression and pooling, I get about 10x the raw data being archived with backuppc - it beats juggling tapes and you can let the users access the backups of their own machine through a web interface. There are some down sides to plan around though: the compression takes some CPU and is slower than a stock rsync run, and the pooling is done with hardlinks which forces the archive to be on a single filesystem and makes it hard to duplicate for offsite copies. There's an RPM in epel that is easy to install on Centos. Sound very interesting indeed! I don't think the performance will be a problem, the server's a calculation machine that has now been scrapped running a dual-x...@2,something GHz and some 4GB RAM IIRC. Do you think the software-raid5 array used, would be a problem in this case? I've never had any problems with software raid5 in linux before, but you never know... -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Alan McKay Sent: Wednesday, January 13, 2010 3:01 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server BackupPC over here - very happy with it for Linux and Windoze, at home and work My google searches would have me believe that Amanda is the more popular choice for backup on linux. On this list it seems Backuppc is. Strange... ;-) -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
- Original Message From: Les Mikesell lesmikes...@gmail.com To: CentOS mailing list centos@centos.org Sent: Wed, January 13, 2010 2:49:11 PM Subject: Re: [CentOS] Backup server Sorin Srbu wrote: -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Barry Brimer Sent: Wednesday, January 13, 2010 2:15 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server I use rsnapshot .. which manages sets of rsync backups using hardlinks. IT may be similar to what you are using already .. Somewhat similar, thanks. I think however I need to get away from this sort of backups. They're just to space-consuming. Between compression and pooling, I get about 10x the raw data being archived with backuppc - it beats juggling tapes and you can let the users access the backups of their own machine through a web interface. There are some down sides to plan around though: the compression takes some CPU and is slower than a stock rsync run, and the pooling is done with hardlinks which forces the archive to be on a single filesystem and makes it hard to duplicate for offsite copies. There's an RPM in epel that is easy to install on Centos. One thing that made me not use BackupPC was that (from the doc): The advantage of the mod_perl setup is that no setuid script is needed, and there is a huge performance advantage The typical speedup is around 15 times. To use mod_perl you need to run Apache as user __BACKUPPCUSER__. If you need to run multiple Apache's for different services then you need to create multiple top-level Apache directories, each with their own config file. You can make copies of /etc/init.d/httpd and use the -d option to httpd to point each http to a different top-level directory. Or you can use the -f option to explicitly point to the config file. Multiple Apache's will run on different Ports (eg: 80 is standard, 8080 is a typical alternative port accessed via http://yourhost.com:8080). Since I don't have a dedicated backup server, I did not want to mess up the existing apache configurations... JD ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
I like better Bacula ... Personal I have Bacula ... configure to backup mac, windows, and linux servers ... Gabe -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Sorin Srbu Sent: Wednesday, January 13, 2010 9:53 AM To: 'CentOS mailing list' Subject: Re: [CentOS] Backup server -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Alan McKay Sent: Wednesday, January 13, 2010 3:01 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server BackupPC over here - very happy with it for Linux and Windoze, at home and work My google searches would have me believe that Amanda is the more popular choice for backup on linux. On this list it seems Backuppc is. Strange... ;-) -- /Sorin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of John Doe Sent: Wednesday, January 13, 2010 4:04 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server [...] If you need to run multiple Apache's for different services then you need to create multiple top-level Apache directories, each with their own config file. You can make copies of /etc/init.d/httpd and use the -d option to httpd to point each http to a different top-level directory. [...] Since I don't have a dedicated backup server, I did not want to mess up the existing apache configurations... Thank for the heads-up, that was actually quite an essential piece of information. In my case however, that won't be a problem as the server would be a dedicated backup-server with no other services running for the department. It might be at home though. If all this folds out well I'll implement it at home as well. -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Gabriel Rosca Sent: Wednesday, January 13, 2010 4:09 PM To: 'CentOS mailing list' Subject: Re: [CentOS] Backup server I like better Bacula ... Personal I have Bacula ... configure to backup mac, windows, and linux servers ... Isn't Bacula payware? -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On 13.1.2010 12:04, Sorin Srbu wrote: Hi all, I've built a new backup server for our linux-clients. Is Amanda the way to go for a backup-solution? It seems to be pretty powerful, if a bit finickety to set up initially. The way we currently do backups is to use rsync from the clients to two folders on an older server that rolls over every other week. This worked fine for a while, but the rsync is cumulative and the users generate a tremendous amount of data every day after having had a client upgrade with newer and hilariously fast computers for calculation. The previous *nix-admin set it up this way with rsync, meaning that we in the long run have data that is way obsolete and get increasingly difficult to maintain. As the backup solution must be next to free, ie free beer, Amanda looks suitable. What do you use for backing up data? I think nobody has yet mentioned rdiff-backup. I have very good experiences with it. Easy to setup and control (only remember first to install the required packages, and I think rsync-devel was not mentioned but is required). Rdiff-backup keeps up an exact copy of the source director(ies), plus it maintains a separate directory for deleted/changed items. With an appropriate command you can restore the source directory as it was at a given point of time. Very neat, and space-saving. http://rdiff-backup.nongnu.org/ - Jussi -- Jussi Hirvi * Green Spot Topeliuksenkatu 15 C * 00250 Helsinki * Finland Tel. +358 9 493 981 * Mobile +358 40 771 2098 (only sms) jussi.hi...@greenspot.fi * http://www.greenspot.fi Remember that if you can't understand an answer, it's okay, we'll just keep going forward as if you do. -- Peter Gulutzan, MySQL 5.0 Stored Procedures ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
They have open source www.bacula.org Gabe -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Sorin Srbu Sent: Wednesday, January 13, 2010 10:20 AM To: 'CentOS mailing list' Subject: Re: [CentOS] Backup server -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Gabriel Rosca Sent: Wednesday, January 13, 2010 4:09 PM To: 'CentOS mailing list' Subject: Re: [CentOS] Backup server I like better Bacula ... Personal I have Bacula ... configure to backup mac, windows, and linux servers ... Isn't Bacula payware? -- /Sorin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On Wed, Jan 13, 2010 at 8:36 AM, Sorin Srbu sorin.s...@orgfarm.uu.se wrote: -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Barry Brimer Sent: Wednesday, January 13, 2010 2:15 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server I use rsnapshot .. which manages sets of rsync backups using hardlinks. IT may be similar to what you are using already .. Somewhat similar, thanks. I think however I need to get away from this sort of backups. They're just to space-consuming. -- /Sorin Do you currently need that disk space for something else? If not, then there is no reason to get rid of the old files. An empty disk is a wasted disk, so you would be creating free space on a perfectly good disk that you already paid for. Having blank space gains you nothing, but deleting a file that you could need in 6 months might cost you. Pick a percentage of the disk that you are comfortable using for backups, and then let the backups grow to that size. Only when you reach that size or need the space for something with a higher priority should you worry about recovering the space. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Sorin Srbu wrote: Sound very interesting indeed! I don't think the performance will be a problem, the server's a calculation machine that has now been scrapped running a dual-x...@2,something GHz and some 4GB RAM IIRC. Do you think the software-raid5 array used, would be a problem in this case? I've never had any problems with software raid5 in linux before, but you never know... The reason I really like BackupPC is the compression you can get. It really helps me out since I have a smaller dedicated backup server. For my instance, since I have a dedicated server, I'd trade performance for the space compression saves me. Regards, Max ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On 1/13/2010 9:08 AM, Gabriel Rosca wrote: My google searches would have me believe that Amanda is the more popular choice for backup on linux. On this list it seems Backuppc is. Strange... ;-) Amanda is good for tape, and has a nice feature of being able to estimate the sizes of full and incremental runs ahead of time and make adjustments to make them all fit on the available tape. It can save to disk but doesn't do any pooling. Backuppc is mostly designed for online backups. It can archive tar images out to tape but it's an afterthought and not great at it. Both are pretty much 'set up and forget' programs although with amanda you do have to swap the tape every day. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On 1/13/2010 9:04 AM, John Doe wrote: One thing that made me not use BackupPC was that (from the doc): The advantage of the mod_perl setup is that no setuid script is needed, and there is a huge performance advantage The typical speedup is around 15 times. To use mod_perl you need to run Apache as user __BACKUPPCUSER__. If you need to run multiple Apache's for different services then you need to create multiple top-level Apache directories, each with their own config file. You can make copies of /etc/init.d/httpd and use the -d option to httpd to point each http to a different top-level directory. Or you can use the -f option to explicitly point to the config file. Multiple Apache's will run on different Ports (eg: 80 is standard, 8080 is a typical alternative port accessed via http://yourhost.com:8080). Since I don't have a dedicated backup server, I did not want to mess up the existing apache configurations... You really don't spend any time in the web interface which is the only thing affected by this. And it is fast enough when run as a normal CGI anyway. Try it without mod_perl. You'd also have the option of running backuppc as apache, but that is less secure if other web admins have access to the machine. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On 1/13/2010 8:51 AM, Sorin Srbu wrote: Between compression and pooling, I get about 10x the raw data being archived with backuppc - it beats juggling tapes and you can let the users access the backups of their own machine through a web interface. There are some down sides to plan around though: the compression takes some CPU and is slower than a stock rsync run, and the pooling is done with hardlinks which forces the archive to be on a single filesystem and makes it hard to duplicate for offsite copies. There's an RPM in epel that is easy to install on Centos. Sound very interesting indeed! I don't think the performance will be a problem, the server's a calculation machine that has now been scrapped running a dual-x...@2,something GHz and some 4GB RAM IIRC. Do you think the software-raid5 array used, would be a problem in this case? I've never had any problems with software raid5 in linux before, but you never know... There's a big write performance hit from raid5 (software or not). It may not be enough to be a showstopper but I wouldn't recommend it. Can you reconfigure to a 0+1 or some other type that has better performance without losing too much space? The archive does have to be on a single filesystem, though, and if you use the epel RPM it makes things easier if you mount the volume at /var/lib/BackupPC before the install. See 'how to change archive directory' from: http://sourceforge.net/apps/mediawiki/backuppc/index.php?title=Tips_and_Tricks to do it after the install. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On Wed, 13 Jan 2010, Gabriel Rosca wrote: I like better Bacula ... Personal I have Bacula ... configure to backup mac, windows, and linux servers ... +1 We back up the same mix with Bacula: Linux, Mac, Windows. We're still using tape for off-site backups, which bacula handles quite well. I would probably revisit my software choice if I started using only HD-based backups. -- Paul Heinlein heinl...@madboa.com http://www.madboa.com/ ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
On Wed, 2010-01-13 at 10:03 -0600, Les Mikesell wrote: On 1/13/2010 9:04 AM, John Doe wrote: One thing that made me not use BackupPC was that (from the doc): The advantage of the mod_perl setup is that no setuid script is needed, and there is a huge performance advantage The typical speedup is around 15 times. Since I don't have a dedicated backup server, I did not want to mess up the existing apache configurations... You really don't spend any time in the web interface which is the only thing affected by this. And it is fast enough when run as a normal CGI anyway. Try it without mod_perl. You'd also have the option of running backuppc as apache, but that is less secure if other web admins have access to the machine. As a side note, the epel BackupPC package does NOT use mod_perl by default and the centos-testing package does use mod_perl by default. I run the centos-testing package (with mod_perl) and the epel package with and without mod_perl usage and see no practical advantage of using BackupPC with mod_perl in terms of time/cycle usage. So just use the stock epel package and you don't need to modify apache. Steve ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
Jussi Hirvi wrote: I think nobody has yet mentioned rdiff-backup. I have very good experiences with it. Easy to setup and control (only remember first to install the required packages, and I think rsync-devel was not mentioned but is required). What did you run into that requires rsync-devel? I've been running rdiff-backup quite happily for about 6 months without having rsync-devel installed either on the server or any of the clients (CentOS 5 server; mix of CentOS 5 and Fedora 12 clients). rsync-2.6.8-3.1.i386.rpm rdiff-backup-1.2.8-1.el5.rf.i386.rpm (from rpmforge) -- Bob Nichols NOSPAM is really part of my email address. Do NOT delete it. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Gabriel Rosca Sent: Wednesday, January 13, 2010 4:21 PM To: 'CentOS mailing list' Subject: Re: [CentOS] Backup server They have open source www.bacula.org My mistake. Thanks for the info! -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup server
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Brian Mathis Sent: Wednesday, January 13, 2010 4:32 PM To: CentOS mailing list Subject: Re: [CentOS] Backup server Do you currently need that disk space for something else? No, I don't. This'd be a dedicated backup server. If not, then there is no reason to get rid of the old files. An empty disk is a wasted disk, so you would be creating free space on a perfectly good disk that you already paid for. Having blank space gains you nothing, but deleting a file that you could need in 6 months might cost you. Pick a percentage of the disk that you are comfortable using for backups, and then let the backups grow to that size. Only when you reach that size or need the space for something with a higher priority should you worry about recovering the space. Valid points indeed. The linux users here create about 50-100GB of data per day. Not all of that data is needed in six months of course, as most of it is only transient and is afterwards put through some other program(s) and so on. Add to that a user reluctance to actually delete old un-needed data and me not knowing when/if I actually can delete stuff, and 2TB will fill up pretty quick. In the best of worlds these two terabytes I have available will be enough for a year. Hopefully longer... 8-/ To summarise, in my environment, it's not actually the diskspace filling up that is the real problem, it's how *fast* it fills up. I'm kinda' trying to both eat the cookie and have it left, as it were. 8-} -- /Sorin smime.p7s Description: S/MIME cryptographic signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
