[Veritas-bu] NetApp VTL Direct Tape Creation and NetBackup
Hey Dan C., I was going to start looking into that exact same setup. What version of netbackup and netapp vtl were you using? I'm currently using direct tape copy but the issue we have now is that too many tapes are going offsite. So i was going to utilize SLPs and use the VTL as the data mover like you were saying to help resolve this without completely restructuring my netbackup policies. My concern is that if i do this, it's going to have to be a cut-over because we want the vtl to be reformatted into a filer at the same time. This way i can pave the way for site to site replication between filers and eliminate tape completely. Anyways, would you mind going into what kinds of issues you were having? It would be greatly appreciated. Thanks! Matt +-- |This was sent by matt.schnei...@mybrighthouse.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] NetApp VTL Direct Tape Creation and NetBackup
I can confirm that this is correct. -Original Message- From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of dan1home Sent: Wednesday, November 18, 2009 3:15 PM To: VERITAS-BU@MAILMAN.ENG.AUBURN.EDU Subject: [Veritas-bu] NetApp VTL Direct Tape Creation and NetBackup I spoke to a colleage of mine who is familiar with the Netapp VTL device and he told me that the limitation spoken of on the Netapp VTL is incorrect. The only 1 to 1 corelation between the Netapp VTL and the physical tape library are the physical tapes not the drives. Have you discussed this with Netapp at all? +-- |This was sent by danny...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu This e-mail and any attachments may contain confidential information of Northwestern Mutual. If you are not the intended recipient of this message, be aware that any disclosure, copying, distribution or use of this e-mail and any attachments is prohibited. If you have received this e-mail in error, please notify Northwestern Mutual immediately by returning it to the sender and delete all copies from your system. Please be advised that communications received via the Northwestern Mutual Secure Message Center are secure. Communications that are not received via the Northwestern Mutual Secure Message Center may not be secure and could be observed by a third party. Thank you for your cooperation. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
[Veritas-bu] NetApp VTL Direct Tape Creation and NetBackup
I spoke to a colleage of mine who is familiar with the Netapp VTL device and he told me that the limitation spoken of on the Netapp VTL is incorrect. The only 1 to 1 corelation between the Netapp VTL and the physical tape library are the physical tapes not the drives. Have you discussed this with Netapp at all? +-- |This was sent by danny...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] NetApp VTL Direct Tape Creation and NetBackup
Agreed, great thread We currently have a SUN VTL that has NDMP functionality. i.e. the physical drives are zoned into the vtl only. Add the drives are configured as NDMP drives. The idea was to utilise SLP's for the primary write to the vtl and duplicate off to physical with the VTL acting as the data mover. In theory all this was great but it turns out that the VTL NDMP component is incredibly flakey and we have now had to resort back to traditional methods. Regards David Clooney From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of Conner, Neil Sent: 26 October 2009 22:47 To: Ed Wilts; hkyeak...@gmail.com Cc: Veritas List Subject: Re: [Veritas-bu] NetApp VTL Direct Tape Creation and NetBackup This is a great analysis... However, one thing a VTL is good for is handling slow clients. For me, backing up about a hundred clients to 12 virtual tape drives significantly shortened my backup window compared to backing up to 4 LTO3 drives (I have quite a few slow clients to contend with so backing up straight to the LTO3 drives was never an option anyway). I didn't like how NetApp implemented their direct to tape feature so I use Vault (soon to be Storage Lifecycle Policies) to duplicate images from the VTL to physical tape. I set aside a 5th LTO3 drive for restores. I get great throughput with the Vault jobs and it's pretty much trouble free. Neil On 10/26/09 11:42 AM, Ed Wilts ewi...@ewilts.org wrote: On Mon, Oct 26, 2009 at 1:00 PM, Heathe Kyle Yeakley hkyeak...@gmail.com wrote: Here's my current layout: Hardware: Spectra Logic T380 with 12 IBM LTO4 tape drives. * I'm told the theoretical bandwidth of an LTO4 tape drive is approximately 120 MB/s. If I have 12 drives, I'm assuming I can say that my library should theoretically be able to handle data at (12 x 120 MB/s) 1,440 MB/s. Not quite true. You can theoretically deliver UNCOMPRESSED data to tape at 1.4GB/sec. However, if you are getting 2:1 compression, you can deliver twice that rate - 2.88GB/sec. If you're getting 3:1 compression, you can deliver 360MB/sec per tape drive. That's 1 dedicated 4Gbps fiber channel port per drive. For 12 drives, you can possible write at 4.3GB/sec depending on your compression ratio. NetBackup: 1 Master (linux), 2 Media (linux), and 3 San Media Servers (Tru64). I suspect you don't have enough media servers to be able to deliver the data rate to keep the tape drives busy. * I have a library that can receive data much faster than my network can deliver it. Welcome to the club :-( The first misunderstanding is that disk is faster than tape. In general, it isn't. You'd be hard pressed to find a disk subsystem that your management is willing to pay for that can actually write at 4.3GB/sec. Management typically thinks that just because they're backups, you can use cheap (SATA) disks without really acknowledging that backups can have the highest I/O workloads of any application that the company runs. Assuming your master server does no tape I/O, you have 5 media servers to put data to tape. That's 2-3 tape drives per media server. You will need at an absolute minimum a pair of HBAs dedicated to tape work - and pray that NetBackup actually will give you 1 tape drive per HBA, which it doesn't even try to do - and you'll need to receive at least 240MB/sec of network traffic - i.e. 3 GigE connections for those non-SAN media servers. Performance management is tough and unless you can clearly identify the bottlenecks you have now, buying hardware is NOT the right answer. You may very likely be throwing money at the wrong problem. If, for example, you don't have enough media servers, you'll have wasted the money on the VTLs. Without de-dupe and assuming 2:1 compression, the VTL 1400 can injest 8.2TB per hour. That's 2.3GB/sec and is actually *LESS* than what your 12 LTO-4 drives are capable of with 2:1 compression (2.8GB/sec). With de-dupe running, you'll get about half that performance out of the VTL (4.3TB/hr) On top of all that, the direct tape creation speed is rated at 3.0TB/hr. That's 0.8GB/sec and now you're significantly less than what your existing tape drives are capable of. why did I buy a VTL? I haven't gained anything from what I see. It depends on the problem you're trying to solve. Assumptions: One of the principle reasons anyone deploys a VTL is because: * Disk is faster than tape at the expense of disk not being removeable and having a lower Mean Time Between Failures than tape. * Backup windows are shrinking. A VTL allows you to create several virtual drives that allow you to write more backups concurrently, thus shrinking your backup window. Both assumptions are actually false. Disk is not faster than tape (see the math above) and although your backup windows are shrinking, a VTL may not allow you to write more backups concurrently. VTLs may solve
[Veritas-bu] NetApp VTL Direct Tape Creation and NetBackup
I am deploying a NetApp VTL 1400 (VTL OS Version 6.0) at my local site. I am working with the NetApp engineer assigned to our deployment to layout which policies are written to which virtual library, etc. The topic of Direct Tape Creation came up and I'm getting some information that is greatly confusing me, and I wanted to blast a message out to the board here to get some feedback. First, an quick idea of my layout and assumptions that I'm making about why we bought the VTL in the first place. Assumptions: One of the principle reasons anyone deploys a VTL is because: * Disk is faster than tape at the expense of disk not being removeable and having a lower Mean Time Between Failures than tape. * Backup windows are shrinking. A VTL allows you to create several virtual drives that allow you to write more backups concurrently, thus shrinking your backup window. Here's my current layout: Hardware: Spectra Logic T380 with 12 IBM LTO4 tape drives. * I'm told the theoretical bandwidth of an LTO4 tape drive is approximately 120 MB/s. If I have 12 drives, I'm assuming I can say that my library should theoretically be able to handle data at (12 x 120 MB/s) 1,440 MB/s. I currently have a performance bottleneck in that I have a couple hundred clients that are backing up over the LAN. Any one client writes at approximately 25 MB/s to 45 MB/s. So I have a physical library that can receive data faster than my LAN can pump it. I've addressed this with management and we're considering some technologies to increase our client's ability to pump data faster to the Spectra Logic library. Policies: I have my policies staggered throughout the night. I have a batch that kicks off at 6pm, another batch that kicks off at 8pm, 10pm, Midnight and 2 am. NetBackup: 1 Master (linux), 2 Media (linux), and 3 San Media Servers (Tru64). Here's the layout I have in my head for the VTL. __ ___ _ | NetBackup | --- | SAN | - | VTL | | Spectra Logic | --- --- --- In other words, I want to zone the VTL so that it's the only library seen by NetBackup, and then have my Spectra Logic zoned so that it's behind my VTL. Furthermore, I've read Symantec's white paper on VTL's and they recommend *NOT* using Shared Storage Option (SSO) with a VTL. So I essentially want to make three virtual libraries, each with maybe 20 or 30 drives, and present each library to my Master and two Media servers, like this: Master Server Virtual Library 1 (This library has 20 to 30 drives and is zoned so that it is only seen by the Master Server). Media Server 1 Virtual Library 2 (This library has 20 to 30 drives and is zoned so that it is only seen by the first Media Server). Media Server 2 - Virtual Library 3 (This library has 20 to 30 drives and is zoned so that it is only seen by the second Media Server). With the 60 to 90 drives that would provide, I could setup a storage unit for each library, and then each night just start ALL my policies and 6 pm and each one would get a drive and begin writing. This configuration would definitely shrink my backup window. However ... (there's always a catch, isn't there?) I want to use Direct Tape Creation so that when I come in in the morning, I can write all of last night's backups out to physical media to be taken offsite. My NetApp engineer tells me that when you enable Direct Tape Creation on a Virtual Library, that the Virtual Library has to have a 1-to-1 relationship with the physical library behind it. In other words, since my physical library has 12 tape drives and 1 robot, my VTL is limited to having 12 tape drives and 1 robot. If that's true, then I'm completely confused about what the point was in buying the VTL. I thought I'd be able to: * Setup the configuration I described previously. * Set all my backup policies to kick off at 6 pm. With 60 to 90 virtual drives available, most if not all of my clients would get a tape drive to write to. The rest would queue up and wait for a resource. I say good bye to status code 196. * I come in in the morning, log into my VTL and kick off the operation to write last night's backups from virtual tape to physical tape. My Spectra Logic only has 12 physical drives, so the first 12 virtual tapes that needed to be written to physical tape would get a physical drive to write to, and the other 50+ tapes would just queue up on the VTL and wait for a free LTO4 drive in the Spectra Logic to become available. If what my NetApp engineer is telling me is correct, I'm only going to be able to present a single virtual library, consisting of 12 virtual drives, to NetBackup. So now I'm back to my
Re: [Veritas-bu] NetApp VTL Direct Tape Creation and NetBackup
On Mon, Oct 26, 2009 at 1:00 PM, Heathe Kyle Yeakley hkyeak...@gmail.comwrote: Here's my current layout: Hardware: Spectra Logic T380 with 12 IBM LTO4 tape drives. * I'm told the theoretical bandwidth of an LTO4 tape drive is approximately 120 MB/s. If I have 12 drives, I'm assuming I can say that my library should theoretically be able to handle data at (12 x 120 MB/s) 1,440 MB/s. Not quite true. You can theoretically deliver UNCOMPRESSED data to tape at 1.4GB/sec. However, if you are getting 2:1 compression, you can deliver twice that rate - 2.88GB/sec. If you're getting 3:1 compression, you can deliver 360MB/sec per tape drive. That's 1 dedicated 4Gbps fiber channel port per drive. For 12 drives, you can possible write at 4.3GB/sec depending on your compression ratio. NetBackup: 1 Master (linux), 2 Media (linux), and 3 San Media Servers (Tru64). I suspect you don't have enough media servers to be able to deliver the data rate to keep the tape drives busy. * I have a library that can receive data much faster than my network can deliver it. Welcome to the club :-( The first misunderstanding is that disk is faster than tape. In general, it isn't. You'd be hard pressed to find a disk subsystem that your management is willing to pay for that can actually write at 4.3GB/sec. Management typically thinks that just because they're backups, you can use cheap (SATA) disks without really acknowledging that backups can have the highest I/O workloads of any application that the company runs. Assuming your master server does no tape I/O, you have 5 media servers to put data to tape. That's 2-3 tape drives per media server. You will need at an absolute minimum a pair of HBAs dedicated to tape work - and pray that NetBackup actually will give you 1 tape drive per HBA, which it doesn't even try to do - and you'll need to receive at least 240MB/sec of network traffic - i.e. 3 GigE connections for those non-SAN media servers. Performance management is tough and unless you can clearly identify the bottlenecks you have now, buying hardware is NOT the right answer. You may very likely be throwing money at the wrong problem. If, for example, you don't have enough media servers, you'll have wasted the money on the VTLs. Without de-dupe and assuming 2:1 compression, the VTL 1400 can injest 8.2TB per hour. That's 2.3GB/sec and is actually *LESS* than what your 12 LTO-4 drives are capable of with 2:1 compression (2.8GB/sec). With de-dupe running, you'll get about half that performance out of the VTL (4.3TB/hr) On top of all that, the direct tape creation speed is rated at 3.0TB/hr. That's 0.8GB/sec and now you're significantly less than what your existing tape drives are capable of. why did I buy a VTL? I haven't gained anything from what I see. It depends on the problem you're trying to solve. Assumptions: One of the principle reasons anyone deploys a VTL is because: * Disk is faster than tape at the expense of disk not being removeable and having a lower Mean Time Between Failures than tape. * Backup windows are shrinking. A VTL allows you to create several virtual drives that allow you to write more backups concurrently, thus shrinking your backup window. Both assumptions are actually false. Disk is not faster than tape (see the math above) and although your backup windows are shrinking, a VTL may not allow you to write more backups concurrently. VTLs may solve problems, but backup performance in large environments is not one of them. Restore performance is one area where they do help. .../Ed Ed Wilts, RHCE, BCFP, BCSD, SCSP, SCSE ewi...@ewilts.org ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] NetApp VTL Direct Tape Creation and NetBackup
This is a great analysis... However, one thing a VTL is good for is handling slow clients. For me, backing up about a hundred clients to 12 virtual tape drives significantly shortened my backup window compared to backing up to 4 LTO3 drives (I have quite a few slow clients to contend with so backing up straight to the LTO3 drives was never an option anyway). I didn¹t like how NetApp implemented their direct to tape feature so I use Vault (soon to be Storage Lifecycle Policies) to duplicate images from the VTL to physical tape. I set aside a 5th LTO3 drive for restores. I get great throughput with the Vault jobs and it¹s pretty much trouble free. Neil On 10/26/09 11:42 AM, Ed Wilts ewi...@ewilts.org wrote: On Mon, Oct 26, 2009 at 1:00 PM, Heathe Kyle Yeakley hkyeak...@gmail.com wrote: Here's my current layout: Hardware: Spectra Logic T380 with 12 IBM LTO4 tape drives. * I'm told the theoretical bandwidth of an LTO4 tape drive is approximately 120 MB/s. If I have 12 drives, I'm assuming I can say that my library should theoretically be able to handle data at (12 x 120 MB/s) 1,440 MB/s. Not quite true. You can theoretically deliver UNCOMPRESSED data to tape at 1.4GB/sec. However, if you are getting 2:1 compression, you can deliver twice that rate - 2.88GB/sec. If you're getting 3:1 compression, you can deliver 360MB/sec per tape drive. That's 1 dedicated 4Gbps fiber channel port per drive. For 12 drives, you can possible write at 4.3GB/sec depending on your compression ratio. NetBackup: 1 Master (linux), 2 Media (linux), and 3 San Media Servers (Tru64). I suspect you don't have enough media servers to be able to deliver the data rate to keep the tape drives busy. * I have a library that can receive data much faster than my network can deliver it. Welcome to the club :-( The first misunderstanding is that disk is faster than tape. In general, it isn't. You'd be hard pressed to find a disk subsystem that your management is willing to pay for that can actually write at 4.3GB/sec. Management typically thinks that just because they're backups, you can use cheap (SATA) disks without really acknowledging that backups can have the highest I/O workloads of any application that the company runs. Assuming your master server does no tape I/O, you have 5 media servers to put data to tape. That's 2-3 tape drives per media server. You will need at an absolute minimum a pair of HBAs dedicated to tape work - and pray that NetBackup actually will give you 1 tape drive per HBA, which it doesn't even try to do - and you'll need to receive at least 240MB/sec of network traffic - i.e. 3 GigE connections for those non-SAN media servers. Performance management is tough and unless you can clearly identify the bottlenecks you have now, buying hardware is NOT the right answer. You may very likely be throwing money at the wrong problem. If, for example, you don't have enough media servers, you'll have wasted the money on the VTLs. Without de-dupe and assuming 2:1 compression, the VTL 1400 can injest 8.2TB per hour. That's 2.3GB/sec and is actually *LESS* than what your 12 LTO-4 drives are capable of with 2:1 compression (2.8GB/sec). With de-dupe running, you'll get about half that performance out of the VTL (4.3TB/hr) On top of all that, the direct tape creation speed is rated at 3.0TB/hr. That's 0.8GB/sec and now you're significantly less than what your existing tape drives are capable of. why did I buy a VTL? I haven't gained anything from what I see. It depends on the problem you're trying to solve. Assumptions: One of the principle reasons anyone deploys a VTL is because: * Disk is faster than tape at the expense of disk not being removeable and having a lower Mean Time Between Failures than tape. * Backup windows are shrinking. A VTL allows you to create several virtual drives that allow you to write more backups concurrently, thus shrinking your backup window. Both assumptions are actually false. Disk is not faster than tape (see the math above) and although your backup windows are shrinking, a VTL may not allow you to write more backups concurrently. VTLs may solve problems, but backup performance in large environments is not one of them. Restore performance is one area where they do help. .../Ed Ed Wilts, RHCE, BCFP, BCSD, SCSP, SCSE ewi...@ewilts.org ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
