Re: Tape drives -- Recommendations?
On Fri, Oct 17, 2008 at 04:39:05PM +0200, Paul Bijnens wrote: > On 2008-10-17 15:23, Greg Troxel wrote: >> Gene Heskett <[EMAIL PROTECTED]> writes: >> >> There is certainly merit in the hard drive approach, but you can lose >> two critical properties if you aren't careful: >> >> full backups offline (not writable by computer) even while making next >> full backup. >> >> full backups taken to a remote location > > I solve these issues by doing backups to an external harddrive. > > USB-2 works reasonably fast, if the amount of data is not too large. I concur. I back up my SOHO network to an external USB HD. I then rotate two more external USB HDs for off-site backup. Backing to the off-site machines is scripted and uses rsync. The two off-site HDs are encrypted with ecryptfs, and the passwords exist in one place. The backup server is a FIT-PC (http://www.fit-pc.com/new/), so my power costs are zilch, five to maybe 15 watts for the FIT-PC and two external HDs (when they're powered up). Try that with your SCSI tape drives. :-) As an extra added benefit, I can do an integrity check on all my "tapes" with "diff -r --brief". It takes a while, but with screen, who cares? Another benefit is that I can also copy my bare metal restore data onto the same off-site HDs, again using rsync and ecryptfs. > > I will be experimenting with eSATA "real soon now". > One of the ideas I have is to make a mirror with LVM of my vtapes > to an external disk for offsite storage. I went with rsync because I know it only copies changes. I don't know whether LVM copies will do that. Also, I can encrypt on the fly while copying to the off-site HDs. I haven't tried that with LVM. > That disk gets exchanged on friday each week, and stored offsite. > Using the USB subsystem makes the server (also small) rather unresponsive. > Only workable in the weekends and nights. I hope eSATA will make > better use of system resources. Interesting. I'm running a diff right now, and top indicates that diff itself is far and away the biggest resource hog. The two USB processes are there but less than diff by two orders of magnitude. Maybe your server has a more CPU intensive USB controller than mine? -- Charles Curley /"\ASCII Ribbon Campaign Looking for fine software \ /Respect for open standards and/or writing? X No HTML/RTF in email http://www.charlescurley.com/ \No M$ Word docs in email Key fingerprint = CE5C 6645 A45A 64E4 94C0 809C FFF6 4C48 4ECD DFDB signature.asc Description: Digital signature
Re: Tape drives -- Recommendations?
On 2008-10-17 15:23, Greg Troxel wrote: Gene Heskett <[EMAIL PROTECTED]> writes: On Wednesday 15 October 2008, Dustin J. Mitchell wrote: On Wed, Oct 15, 2008 at 10:44 AM, Seann Clark <[EMAIL PROTECTED]> wrote: The other alternative I am looking into is getting a large external case and cramming it full of 1TB hard drives and using that as backup, but I would like a tape system that works well. This option is worth considering. HD's have many fewer moving parts than tapes/drives. Even if the "critical failure" rate is similar, the "annoying failure" rate of tape drives is much higher. Which is to say, they require a lot more fiddling. Dustin I'll back Dustin up on this one. Switching to a hard drive got rid of 99.9% of by backup problems. It Just Works(TM). -- Cheers, Gene There is certainly merit in the hard drive approach, but you can lose two critical properties if you aren't careful: full backups offline (not writable by computer) even while making next full backup. full backups taken to a remote location I solve these issues by doing backups to an external harddrive. USB-2 works reasonably fast, if the amount of data is not too large. I will be experimenting with eSATA "real soon now". One of the ideas I have is to make a mirror with LVM of my vtapes to an external disk for offsite storage. I have been using LTO-2 for several years and have had little enough trouble, although I can't remember if it is very little or zero. Before that I had DDS-3 and that was occasionally annoying but not that bad. I am firmly in the tape camp at least for corporate use. But actually, I still use LTO-2 for offsite storage for our main office; it's only in 2 small offices abroad that I implemented backup to external USB disk. That disk gets exchanged on friday each week, and stored offsite. Using the USB subsystem makes the server (also small) rather unresponsive. Only workable in the weekends and nights. I hope eSATA will make better use of system resources. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... "Are you sure?" ... YES ... Phew ... I'm out * ***
Re: Tape drives -- Recommendations?
Gene Heskett <[EMAIL PROTECTED]> writes: > On Wednesday 15 October 2008, Dustin J. Mitchell wrote: >>On Wed, Oct 15, 2008 at 10:44 AM, Seann Clark >> >><[EMAIL PROTECTED]> wrote: >>> The other alternative I am looking into is getting a large external case >>> and cramming it full of 1TB hard drives and using that as backup, but I >>> would like a tape system that works well. >> >>This option is worth considering. HD's have many fewer moving parts >>than tapes/drives. Even if the "critical failure" rate is similar, >>the "annoying failure" rate of tape drives is much higher. Which is >>to say, they require a lot more fiddling. >> >>Dustin > > I'll back Dustin up on this one. Switching to a hard drive got rid of 99.9% > of by backup problems. It Just Works(TM). > > -- > Cheers, Gene There is certainly merit in the hard drive approach, but you can lose two critical properties if you aren't careful: full backups offline (not writable by computer) even while making next full backup. full backups taken to a remote location I have been using LTO-2 for several years and have had little enough trouble, although I can't remember if it is very little or zero. Before that I had DDS-3 and that was occasionally annoying but not that bad. I am firmly in the tape camp at least for corporate use. pgpFez5QC2Z7v.pgp Description: PGP signature
Re: Tape drives -- Recommendations?
On Wednesday 15 October 2008, Dustin J. Mitchell wrote: >On Wed, Oct 15, 2008 at 10:44 AM, Seann Clark > ><[EMAIL PROTECTED]> wrote: >> The other alternative I am looking into is getting a large external case >> and cramming it full of 1TB hard drives and using that as backup, but I >> would like a tape system that works well. > >This option is worth considering. HD's have many fewer moving parts >than tapes/drives. Even if the "critical failure" rate is similar, >the "annoying failure" rate of tape drives is much higher. Which is >to say, they require a lot more fiddling. > >Dustin I'll back Dustin up on this one. Switching to a hard drive got rid of 99.9% of by backup problems. It Just Works(TM). -- Cheers, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) "We'll look into it": By the time the wheels make a full turn, we assume you will have forgotten about it, too.
RE: Tape drives -- Recommendations?
LTO is the most commonly used tape drive technology among Amanda users. When we conducted an extensive survey of Amanda usage, almost 40% of respondents said they use LTO tape drives: http://wiki.zmanda.com/index.php/Results_of_Amanda_Users_Survey_2006 Considering the price difference between LTO4 ($4,600 per drive and $100 per tape) and LTO 3 ($2,700 per drive and $50 per tape), I would go with LTO3 because you get the same price per GB of data on tape, but spend less money upfront with LTO3. You can get an excellent StorageTek SL24 (24 tape slots) autoloader for $7,000 from Sun, and you can try it for 60 days before buying. A similar LTO autoloader from Dell would cost you around $5,000 if you catch a sale. However, Dell does not offer a trial, and people often mention less than positive experience with Dell's support, especially for non-server and OEM-ed products, e.g. printers or autoloaders. Regards - Dmitri Joukovski -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris Hoogendyk Sent: Wednesday, October 15, 2008 9:35 AM To: Seann Clark Cc: [email protected] Subject: Re: Tape drives -- Recommendations? Seann Clark wrote: > All, > >I am fairly new to the list, not so new to Amanda (I usually troll > IRC when I am looking for help half the time) but I want to poll the > group for suggestions to aid in what I am looking into as well. I have > a system currently that has 3.12TB of data, which I would like to > start backing up regularly, and soon will increase that to a larger > number as well. I am looking for a good service tape drive that can > take care of the physical offloading of backups, and that plays well > with Amanda. I have an old SCSI HP SureStore that I can never get to > really back up to (Pity it was a nice drive for the time, esp when I > was maxed at 700GB) though it can read the tapes, write to the tapes > through Amanda, it just dies partially through and freezes up the > drive. What I am after though is a backup system that is tape based. I > would prefer non SCSI, but I can work around that. > > > The other alternative I am looking into is getting a large external > case and cramming it full of 1TB hard drives and using that as backup, > but I would like a tape system that works well. I haven't gotten that > portion to work too well in the past, but since it was a first time > doing it, I am very sure it was a fatal user error that was preventing > it from happening. It has a huge amount to do with budget. Being in a budget conscious department, I settled on a Sony LIB-162A5. Ballpark cost around $5k. Less if you get good discounts. It uses AIT5 tapes (400G native), comes with one drive but can take a second, holds 16 tapes. The somewhat lower cost comes from being a carousel mechanism that is less complex than the typical robots. That also means it is more reliable, but less expandable than the popular lines of robotic libraries. I think most people are going with LTO. I chose AIT because I liked the technology. It isn't as fast as LTO, but it doesn't shoe shine. I hear plenty of horror stories of people who get a really fast LTO drive and find that they aren't getting any throughput. That's actually because the computer they configure to go with it can't maintain the data throughput that the tape needs, so the tape goes into shoe shining, and the throughput drops even further to dismal levels. Of course, if budget is not an issue, and if you understand your hardware configuration well, then you will configure a backup server that has the capacity to pump data to the tape and keep it going. I have no trouble keeping the AIT5 going at its full rated speed. Why not SCSI? Most of the tape libraries are SCSI (either directly SCSI or via SAS or Fibre Channel). Mine is LVD320 SCSI. I'm not sure what alternative you are thinking of. Whatever you choose, you have to think about throughput. Figure out how much data you are planning to transfer and then calculate optimal times. You won't typically get optimal, but it will put you in the right ballpark. Be sure to account for bits versus bytes in the various transfer technologies. Network stuff is going to be bits, internal bus transfers are typically bytes. So, I run Gigabit network and my AIT5 will do 25MBytes. Fortunately, Amanda will smooth the demand over your dump cycle. So, if you are trying to do 3.12TB total, and you break that up into many DLE's, then you may only be averaging 500GB a night or even less, depending on your dump cycle. I'm sure you already know that, but it is a significant part of the calculations and a real advantage over other backup software. Of course, it certainly doesn't hurt to do both vtapes and tapes. I'm a big fan of redundancy, which is why I
RE: Tape drives -- Recommendations?
Seann I use Sun C2 and SL24 autoloaders with LTO3 drives. Operation is flawless - I divide cycles into runs, load tapes, and check mail every day. The drives themselves are Quantum drives. Regards, Sean -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Seann Clark Sent: Wednesday, October 15, 2008 10:45 AM To: [email protected] Subject: Tape drives -- Recommendations? All, I am fairly new to the list, not so new to Amanda (I usually troll IRC when I am looking for help half the time) but I want to poll the group for suggestions to aid in what I am looking into as well. I have a system currently that has 3.12TB of data, which I would like to start backing up regularly, and soon will increase that to a larger number as well. I am looking for a good service tape drive that can take care of the physical offloading of backups, and that plays well with Amanda. I have an old SCSI HP SureStore that I can never get to really back up to (Pity it was a nice drive for the time, esp when I was maxed at 700GB) though it can read the tapes, write to the tapes through Amanda, it just dies partially through and freezes up the drive. What I am after though is a backup system that is tape based. I would prefer non SCSI, but I can work around that. The other alternative I am looking into is getting a large external case and cramming it full of 1TB hard drives and using that as backup, but I would like a tape system that works well. I haven't gotten that portion to work too well in the past, but since it was a first time doing it, I am very sure it was a fatal user error that was preventing it from happening. Thanks in advance, Seann
Re: Tape drives -- Recommendations?
Seann Clark wrote: All, I am fairly new to the list, not so new to Amanda (I usually troll IRC when I am looking for help half the time) but I want to poll the group for suggestions to aid in what I am looking into as well. I have a system currently that has 3.12TB of data, which I would like to start backing up regularly, and soon will increase that to a larger number as well. I am looking for a good service tape drive that can take care of the physical offloading of backups, and that plays well with Amanda. I have an old SCSI HP SureStore that I can never get to really back up to (Pity it was a nice drive for the time, esp when I was maxed at 700GB) though it can read the tapes, write to the tapes through Amanda, it just dies partially through and freezes up the drive. What I am after though is a backup system that is tape based. I would prefer non SCSI, but I can work around that. The other alternative I am looking into is getting a large external case and cramming it full of 1TB hard drives and using that as backup, but I would like a tape system that works well. I haven't gotten that portion to work too well in the past, but since it was a first time doing it, I am very sure it was a fatal user error that was preventing it from happening. It has a huge amount to do with budget. Being in a budget conscious department, I settled on a Sony LIB-162A5. Ballpark cost around $5k. Less if you get good discounts. It uses AIT5 tapes (400G native), comes with one drive but can take a second, holds 16 tapes. The somewhat lower cost comes from being a carousel mechanism that is less complex than the typical robots. That also means it is more reliable, but less expandable than the popular lines of robotic libraries. I think most people are going with LTO. I chose AIT because I liked the technology. It isn't as fast as LTO, but it doesn't shoe shine. I hear plenty of horror stories of people who get a really fast LTO drive and find that they aren't getting any throughput. That's actually because the computer they configure to go with it can't maintain the data throughput that the tape needs, so the tape goes into shoe shining, and the throughput drops even further to dismal levels. Of course, if budget is not an issue, and if you understand your hardware configuration well, then you will configure a backup server that has the capacity to pump data to the tape and keep it going. I have no trouble keeping the AIT5 going at its full rated speed. Why not SCSI? Most of the tape libraries are SCSI (either directly SCSI or via SAS or Fibre Channel). Mine is LVD320 SCSI. I'm not sure what alternative you are thinking of. Whatever you choose, you have to think about throughput. Figure out how much data you are planning to transfer and then calculate optimal times. You won't typically get optimal, but it will put you in the right ballpark. Be sure to account for bits versus bytes in the various transfer technologies. Network stuff is going to be bits, internal bus transfers are typically bytes. So, I run Gigabit network and my AIT5 will do 25MBytes. Fortunately, Amanda will smooth the demand over your dump cycle. So, if you are trying to do 3.12TB total, and you break that up into many DLE's, then you may only be averaging 500GB a night or even less, depending on your dump cycle. I'm sure you already know that, but it is a significant part of the calculations and a real advantage over other backup software. Of course, it certainly doesn't hurt to do both vtapes and tapes. I'm a big fan of redundancy, which is why I run a long tape cycle and have dual holding disks. -- --- Chris Hoogendyk - O__ Systems Administrator c/ /'_ --- Biology & Geology Departments (*) \(*) -- 140 Morrill Science Center ~~ - University of Massachusetts, Amherst <[EMAIL PROTECTED]> --- Erdös 4
Re: Tape drives -- Recommendations?
Chris Hoogendyk wrote: Seann Clark wrote: All, I am fairly new to the list, not so new to Amanda (I usually troll IRC when I am looking for help half the time) but I want to poll the group for suggestions to aid in what I am looking into as well. I have a system currently that has 3.12TB of data, which I would like to start backing up regularly, and soon will increase that to a larger number as well. I am looking for a good service tape drive that can take care of the physical offloading of backups, and that plays well with Amanda. I have an old SCSI HP SureStore that I can never get to really back up to (Pity it was a nice drive for the time, esp when I was maxed at 700GB) though it can read the tapes, write to the tapes through Amanda, it just dies partially through and freezes up the drive. What I am after though is a backup system that is tape based. I would prefer non SCSI, but I can work around that. The other alternative I am looking into is getting a large external case and cramming it full of 1TB hard drives and using that as backup, but I would like a tape system that works well. I haven't gotten that portion to work too well in the past, but since it was a first time doing it, I am very sure it was a fatal user error that was preventing it from happening. It has a huge amount to do with budget. Being in a budget conscious department, I settled on a Sony LIB-162A5. Ballpark cost around $5k. Less if you get good discounts. It uses AIT5 tapes (400G native), comes with one drive but can take a second, holds 16 tapes. The somewhat lower cost comes from being a carousel mechanism that is less complex than the typical robots. That also means it is more reliable, but less expandable than the popular lines of robotic libraries. I think most people are going with LTO. I chose AIT because I liked the technology. It isn't as fast as LTO, but it doesn't shoe shine. I hear plenty of horror stories of people who get a really fast LTO drive and find that they aren't getting any throughput. That's actually because the computer they configure to go with it can't maintain the data throughput that the tape needs, so the tape goes into shoe shining, and the throughput drops even further to dismal levels. Of course, if budget is not an issue, and if you understand your hardware configuration well, then you will configure a backup server that has the capacity to pump data to the tape and keep it going. I have no trouble keeping the AIT5 going at its full rated speed. Why not SCSI? Most of the tape libraries are SCSI (either directly SCSI or via SAS or Fibre Channel). Mine is LVD320 SCSI. I'm not sure what alternative you are thinking of. Whatever you choose, you have to think about throughput. Figure out how much data you are planning to transfer and then calculate optimal times. You won't typically get optimal, but it will put you in the right ballpark. Be sure to account for bits versus bytes in the various transfer technologies. Network stuff is going to be bits, internal bus transfers are typically bytes. So, I run Gigabit network and my AIT5 will do 25MBytes. SCSI, plain jain SCSI. SAS is a good option, Fibre channel is a bit outside. The biggest thing with that is I would add to the cost to get a good SCSI card, since none of my servers, oddly enough, have a SCSI Bus. Almost all of the Tape Drives I have seen are SCSI, so I was trying to look outside of the box. That is the only base on the preference on that. With the network side, I do understand that, even as I have gig networking, I burn only 25MB/s at max across it (strangely enough it is both wired and wireless that burns the speed like that) since I am using backuppc to gather all my computers into one archive/backup, since it took care of a more immediate problem. Fortunately, Amanda will smooth the demand over your dump cycle. So, if you are trying to do 3.12TB total, and you break that up into many DLE's, then you may only be averaging 500GB a night or even less, depending on your dump cycle. I'm sure you already know that, but it is a significant part of the calculations and a real advantage over other backup software. Breaking up full disks and incremental backups with Amanda looks easy and I have gotten it to work in single tape mode prior, so it is pretty good. Now when the tapes are 4GB max... well that sucks with large data over the time, though it isn't like I had a shortage, just no changer. Of course, it certainly doesn't hurt to do both vtapes and tapes. I'm a big fan of redundancy, which is why I run a long tape cycle and have dual holding disks. I have just recently started playing with vtapes, and haven't gotten it to work just right yet, but that is, once again my fault. This server is a little on the redundant side as it is, with Raid5 as the main disk's protection scheme, and I want to get this stuff onto tape as a backup of a backup. So far a
Re: Tape drives -- Recommendations?
On Wed, Oct 15, 2008 at 10:44 AM, Seann Clark <[EMAIL PROTECTED]> wrote: > The other alternative I am looking into is getting a large external case and > cramming it full of 1TB hard drives and using that as backup, but I would > like a tape system that works well. This option is worth considering. HD's have many fewer moving parts than tapes/drives. Even if the "critical failure" rate is similar, the "annoying failure" rate of tape drives is much higher. Which is to say, they require a lot more fiddling. Dustin -- Storage Software Engineer http://www.zmanda.com
