Re: Deduplication/replication options
I'm not sure, but I suspect that IBM is more willing to use better special-bid pricing to a customer who is using both TSM and ProtecTier. I.e., it's a matter of negotiation, not rigid rules about how to measure occupancy. I don't think there is anything in the occupancy calculation algorithm to compensate for ProtecTier deduplication. Perhaps an IBMer or Business Partner who is monitoring this list could chime in on this? At 06:21 AM 7/26/2013, Steven Langdale wrote: >Hello Stefan > >Have you got cases of this? I ask because I have been specifically told by >our rep that any dedupe saving for capacity licensing is TSM dedupe only, >regarless of the backend storage. > > > >On 26 July 2013 09:16, Stefan Folkerts wrote: > >> No, this is correct, IBM does give Protectier (for example) customers an >> advantage with deduplication and factor in the dedup for billing. >> >> >> On Wed, Jul 24, 2013 at 10:18 PM, Colwell, William F. >> wrote: >> >> > Hi Norman, >> > >> > that is incorrect. IBM doesn't care what the hardware is when measuring >> > used capacity >> > in the Suite for Unified Recovery licensing model. >> > >> > A description of the measurement process and the sql to do it is at >> > http://www-01.ibm.com/support/docview.wss?uid=swg21500482 >> > >> > Thanks, >> > >> > Bill Colwell >> > Draper Lab >> > >> > -----Original Message- >> > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of >> > Gee, Norman >> > Sent: Wednesday, July 24, 2013 11:29 AM >> > To: ADSM-L@VM.MARIST.EDU >> > Subject: Re: Deduplication/replication options >> > >> > This why IBM is pushing their VTL solution. IBM will only charge for the >> > net amount using an all IBM solution. At least that is what I was told. >> > >> > -Original Message- >> > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of >> > Loon, EJ van - SPLXM >> > Sent: Tuesday, July 23, 2013 11:59 PM >> > To: ADSM-L@VM.MARIST.EDU >> > Subject: Re: Deduplication/replication options >> > >> > Hi Sergio! >> > Another thing to take into consideration: if you have switched from PVU >> > licensing to sub-capacity licensing in the past: TSM sub-capacity >> > licensing is based on the amount of data stored in your primary pool. If >> > this data is stored on a de-duplicating storage device you will be >> > charged for the gross amount of data. If you are using TSM >> > de-duplication you will have to pay for the de-duplicated amount. This >> > will probably save you a lot of money... >> > Kind regards, >> > Eric van Loon >> > AF/KLM Storage Engineering >> > >> > -Original Message- >> > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of >> > Sergio O. Fuentes >> > Sent: dinsdag 23 juli 2013 19:20 >> > To: ADSM-L@VM.MARIST.EDU >> > Subject: Deduplication/replication options >> > >> > Hello all, >> > >> > We're currently faced with a decision go with a dedupe storage array or >> > with TSM dedupe for our backup storage targets. There are some very >> > critical pros and cons going with one or the other. For example, TSM >> > dedupe will reduce overall network throughput both for backups and >> > replication (source-side dedupe would be used). A dedupe storage array >> > won't do that for backup, but it would be possible if we replicated to >> > an identical array (but TSM replication would be bandwidth intensive). >> > TSM dedupe might not scale as well and may neccessitate more TSM servers >> > to distribute the load. Overall, though, I think the cost of additional >> > servers is way less than what a native dedupe array would cost so I >> > don't think that's a big hit. >> > >> > Replication is key. We have two datacenters where I would love it if TSM >> > replication could be used in order to quickly (still manually, though) >> > activate the replication server for production if necessary. Having a >> > dedupe storage array kind of removes that option, unless we want to >> > replicate the whole rehydrated backup data via TSM. >> > >> > I'm going on and on here, but has anybody had to make a decision to go >> > one way or the other? Would it make sense to do a hybrid deployment >> > (combination of TSM Dedupe and A
Re: Deduplication/replication options
Yes I do but I can not share the names with people outside of my company, sorry. I'll tell you it's a mid sized company with two Protectiers in two locations that replicate, the customer has the entry level TB license model and IBM used the protectier interface to determin the dedup savings for the license if I am not mistaken. It could very well be what Nick wrote but I would tell your IBM rep that you have read about several cases and request them to ask again. On Fri, Jul 26, 2013 at 12:21 PM, Steven Langdale wrote: > Hello Stefan > > Have you got cases of this? I ask because I have been specifically told by > our rep that any dedupe saving for capacity licensing is TSM dedupe only, > regarless of the backend storage. > > > > On 26 July 2013 09:16, Stefan Folkerts wrote: > > > No, this is correct, IBM does give Protectier (for example) customers an > > advantage with deduplication and factor in the dedup for billing. > > > > > > On Wed, Jul 24, 2013 at 10:18 PM, Colwell, William F. > > wrote: > > > > > Hi Norman, > > > > > > that is incorrect. IBM doesn't care what the hardware is when > measuring > > > used capacity > > > in the Suite for Unified Recovery licensing model. > > > > > > A description of the measurement process and the sql to do it is at > > > http://www-01.ibm.com/support/docview.wss?uid=swg21500482 > > > > > > Thanks, > > > > > > Bill Colwell > > > Draper Lab > > > > > > -Original Message----- > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf > Of > > > Gee, Norman > > > Sent: Wednesday, July 24, 2013 11:29 AM > > > To: ADSM-L@VM.MARIST.EDU > > > Subject: Re: Deduplication/replication options > > > > > > This why IBM is pushing their VTL solution. IBM will only charge for > the > > > net amount using an all IBM solution. At least that is what I was > told. > > > > > > -Original Message- > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf > Of > > > Loon, EJ van - SPLXM > > > Sent: Tuesday, July 23, 2013 11:59 PM > > > To: ADSM-L@VM.MARIST.EDU > > > Subject: Re: Deduplication/replication options > > > > > > Hi Sergio! > > > Another thing to take into consideration: if you have switched from PVU > > > licensing to sub-capacity licensing in the past: TSM sub-capacity > > > licensing is based on the amount of data stored in your primary pool. > If > > > this data is stored on a de-duplicating storage device you will be > > > charged for the gross amount of data. If you are using TSM > > > de-duplication you will have to pay for the de-duplicated amount. This > > > will probably save you a lot of money... > > > Kind regards, > > > Eric van Loon > > > AF/KLM Storage Engineering > > > > > > -Original Message- > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf > Of > > > Sergio O. Fuentes > > > Sent: dinsdag 23 juli 2013 19:20 > > > To: ADSM-L@VM.MARIST.EDU > > > Subject: Deduplication/replication options > > > > > > Hello all, > > > > > > We're currently faced with a decision go with a dedupe storage array or > > > with TSM dedupe for our backup storage targets. There are some very > > > critical pros and cons going with one or the other. For example, TSM > > > dedupe will reduce overall network throughput both for backups and > > > replication (source-side dedupe would be used). A dedupe storage array > > > won't do that for backup, but it would be possible if we replicated to > > > an identical array (but TSM replication would be bandwidth intensive). > > > TSM dedupe might not scale as well and may neccessitate more TSM > servers > > > to distribute the load. Overall, though, I think the cost of > additional > > > servers is way less than what a native dedupe array would cost so I > > > don't think that's a big hit. > > > > > > Replication is key. We have two datacenters where I would love it if > TSM > > > replication could be used in order to quickly (still manually, though) > > > activate the replication server for production if necessary. Having a > > > dedupe storage array kind of removes that option, unless we want to > > > replicate the whole rehydrated backup data via TSM. > > > > > > I'm going on and on her
Re: Deduplication/replication options
On Jul 26, 2013, at 5:21 AM, Steven Langdale wrote: > Hello Stefan > > Have you got cases of this? I ask because I have been specifically told by > our rep that any dedupe saving for capacity licensing is TSM dedupe only, > regarless of the backend storage. During our last TSM license renewal negotiations, we were also undergoing a refresh of our storage for TSM. We heard the same thing Stefan heard: use-based pricing could be reduced by using TSM deduplication or using deduplication in ProtecTier, but there would be no reduction for deduplication effects in non-IBM hardware, such as the DataDomains we use. On the other hand, IBM often reminds us that terms and conditions may vary from region to region and probably from customer to customer, so it's entirely possible that all of us are telling the truth as it applies to us and that the only answer that works for everyone is, "See the company you license TSM from or through." However, that conversation may go better for you if you know what others live with. Nick
Re: Deduplication/replication options
Hello Stefan Have you got cases of this? I ask because I have been specifically told by our rep that any dedupe saving for capacity licensing is TSM dedupe only, regarless of the backend storage. On 26 July 2013 09:16, Stefan Folkerts wrote: > No, this is correct, IBM does give Protectier (for example) customers an > advantage with deduplication and factor in the dedup for billing. > > > On Wed, Jul 24, 2013 at 10:18 PM, Colwell, William F. > wrote: > > > Hi Norman, > > > > that is incorrect. IBM doesn't care what the hardware is when measuring > > used capacity > > in the Suite for Unified Recovery licensing model. > > > > A description of the measurement process and the sql to do it is at > > http://www-01.ibm.com/support/docview.wss?uid=swg21500482 > > > > Thanks, > > > > Bill Colwell > > Draper Lab > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of > > Gee, Norman > > Sent: Wednesday, July 24, 2013 11:29 AM > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: Deduplication/replication options > > > > This why IBM is pushing their VTL solution. IBM will only charge for the > > net amount using an all IBM solution. At least that is what I was told. > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of > > Loon, EJ van - SPLXM > > Sent: Tuesday, July 23, 2013 11:59 PM > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: Deduplication/replication options > > > > Hi Sergio! > > Another thing to take into consideration: if you have switched from PVU > > licensing to sub-capacity licensing in the past: TSM sub-capacity > > licensing is based on the amount of data stored in your primary pool. If > > this data is stored on a de-duplicating storage device you will be > > charged for the gross amount of data. If you are using TSM > > de-duplication you will have to pay for the de-duplicated amount. This > > will probably save you a lot of money... > > Kind regards, > > Eric van Loon > > AF/KLM Storage Engineering > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of > > Sergio O. Fuentes > > Sent: dinsdag 23 juli 2013 19:20 > > To: ADSM-L@VM.MARIST.EDU > > Subject: Deduplication/replication options > > > > Hello all, > > > > We're currently faced with a decision go with a dedupe storage array or > > with TSM dedupe for our backup storage targets. There are some very > > critical pros and cons going with one or the other. For example, TSM > > dedupe will reduce overall network throughput both for backups and > > replication (source-side dedupe would be used). A dedupe storage array > > won't do that for backup, but it would be possible if we replicated to > > an identical array (but TSM replication would be bandwidth intensive). > > TSM dedupe might not scale as well and may neccessitate more TSM servers > > to distribute the load. Overall, though, I think the cost of additional > > servers is way less than what a native dedupe array would cost so I > > don't think that's a big hit. > > > > Replication is key. We have two datacenters where I would love it if TSM > > replication could be used in order to quickly (still manually, though) > > activate the replication server for production if necessary. Having a > > dedupe storage array kind of removes that option, unless we want to > > replicate the whole rehydrated backup data via TSM. > > > > I'm going on and on here, but has anybody had to make a decision to go > > one way or the other? Would it make sense to do a hybrid deployment > > (combination of TSM Dedupe and Array dedupe)? Any thoughts or tales of > > woes and forewarnings are appreciated. > > > > Thanks! > > Sergio > > > > For information, services and offers, please visit our web site: > > http://www.klm.com. This e-mail and any attachment may contain > > confidential and privileged material intended for the addressee only. If > > you are not the addressee, you are notified that no part of the e-mail or > > any attachment may be disclosed, copied or distributed, and that any > other > > action related to this e-mail or attachment is strictly prohibited, and > may > > be unlawful. If you have received this e-mail by error, please notify the > > sender immediately by return e-mail, and delete this message. > > > > Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its > > employees shall not be liable for the incorrect or incomplete > transmission > > of this e-mail or any attachments, nor responsible for any delay in > receipt. > > Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch > > Airlines) is registered in Amstelveen, The Netherlands, with registered > > number 33014286 > > > > > > >
Re: Deduplication/replication options
No, this is correct, IBM does give Protectier (for example) customers an advantage with deduplication and factor in the dedup for billing. On Wed, Jul 24, 2013 at 10:18 PM, Colwell, William F. wrote: > Hi Norman, > > that is incorrect. IBM doesn't care what the hardware is when measuring > used capacity > in the Suite for Unified Recovery licensing model. > > A description of the measurement process and the sql to do it is at > http://www-01.ibm.com/support/docview.wss?uid=swg21500482 > > Thanks, > > Bill Colwell > Draper Lab > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of > Gee, Norman > Sent: Wednesday, July 24, 2013 11:29 AM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: Deduplication/replication options > > This why IBM is pushing their VTL solution. IBM will only charge for the > net amount using an all IBM solution. At least that is what I was told. > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of > Loon, EJ van - SPLXM > Sent: Tuesday, July 23, 2013 11:59 PM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: Deduplication/replication options > > Hi Sergio! > Another thing to take into consideration: if you have switched from PVU > licensing to sub-capacity licensing in the past: TSM sub-capacity > licensing is based on the amount of data stored in your primary pool. If > this data is stored on a de-duplicating storage device you will be > charged for the gross amount of data. If you are using TSM > de-duplication you will have to pay for the de-duplicated amount. This > will probably save you a lot of money... > Kind regards, > Eric van Loon > AF/KLM Storage Engineering > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of > Sergio O. Fuentes > Sent: dinsdag 23 juli 2013 19:20 > To: ADSM-L@VM.MARIST.EDU > Subject: Deduplication/replication options > > Hello all, > > We're currently faced with a decision go with a dedupe storage array or > with TSM dedupe for our backup storage targets. There are some very > critical pros and cons going with one or the other. For example, TSM > dedupe will reduce overall network throughput both for backups and > replication (source-side dedupe would be used). A dedupe storage array > won't do that for backup, but it would be possible if we replicated to > an identical array (but TSM replication would be bandwidth intensive). > TSM dedupe might not scale as well and may neccessitate more TSM servers > to distribute the load. Overall, though, I think the cost of additional > servers is way less than what a native dedupe array would cost so I > don't think that's a big hit. > > Replication is key. We have two datacenters where I would love it if TSM > replication could be used in order to quickly (still manually, though) > activate the replication server for production if necessary. Having a > dedupe storage array kind of removes that option, unless we want to > replicate the whole rehydrated backup data via TSM. > > I'm going on and on here, but has anybody had to make a decision to go > one way or the other? Would it make sense to do a hybrid deployment > (combination of TSM Dedupe and Array dedupe)? Any thoughts or tales of > woes and forewarnings are appreciated. > > Thanks! > Sergio > > For information, services and offers, please visit our web site: > http://www.klm.com. This e-mail and any attachment may contain > confidential and privileged material intended for the addressee only. If > you are not the addressee, you are notified that no part of the e-mail or > any attachment may be disclosed, copied or distributed, and that any other > action related to this e-mail or attachment is strictly prohibited, and may > be unlawful. If you have received this e-mail by error, please notify the > sender immediately by return e-mail, and delete this message. > > Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its > employees shall not be liable for the incorrect or incomplete transmission > of this e-mail or any attachments, nor responsible for any delay in receipt. > Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch > Airlines) is registered in Amstelveen, The Netherlands, with registered > number 33014286 > > >
Re: Deduplication/replication options
Hi Norman, that is incorrect. IBM doesn't care what the hardware is when measuring used capacity in the Suite for Unified Recovery licensing model. A description of the measurement process and the sql to do it is at http://www-01.ibm.com/support/docview.wss?uid=swg21500482 Thanks, Bill Colwell Draper Lab -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Gee, Norman Sent: Wednesday, July 24, 2013 11:29 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: Deduplication/replication options This why IBM is pushing their VTL solution. IBM will only charge for the net amount using an all IBM solution. At least that is what I was told. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Loon, EJ van - SPLXM Sent: Tuesday, July 23, 2013 11:59 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: Deduplication/replication options Hi Sergio! Another thing to take into consideration: if you have switched from PVU licensing to sub-capacity licensing in the past: TSM sub-capacity licensing is based on the amount of data stored in your primary pool. If this data is stored on a de-duplicating storage device you will be charged for the gross amount of data. If you are using TSM de-duplication you will have to pay for the de-duplicated amount. This will probably save you a lot of money... Kind regards, Eric van Loon AF/KLM Storage Engineering -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Sergio O. Fuentes Sent: dinsdag 23 juli 2013 19:20 To: ADSM-L@VM.MARIST.EDU Subject: Deduplication/replication options Hello all, We're currently faced with a decision go with a dedupe storage array or with TSM dedupe for our backup storage targets. There are some very critical pros and cons going with one or the other. For example, TSM dedupe will reduce overall network throughput both for backups and replication (source-side dedupe would be used). A dedupe storage array won't do that for backup, but it would be possible if we replicated to an identical array (but TSM replication would be bandwidth intensive). TSM dedupe might not scale as well and may neccessitate more TSM servers to distribute the load. Overall, though, I think the cost of additional servers is way less than what a native dedupe array would cost so I don't think that's a big hit. Replication is key. We have two datacenters where I would love it if TSM replication could be used in order to quickly (still manually, though) activate the replication server for production if necessary. Having a dedupe storage array kind of removes that option, unless we want to replicate the whole rehydrated backup data via TSM. I'm going on and on here, but has anybody had to make a decision to go one way or the other? Would it make sense to do a hybrid deployment (combination of TSM Dedupe and Array dedupe)? Any thoughts or tales of woes and forewarnings are appreciated. Thanks! Sergio For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 33014286
Re: Deduplication/replication options
This why IBM is pushing their VTL solution. IBM will only charge for the net amount using an all IBM solution. At least that is what I was told. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Loon, EJ van - SPLXM Sent: Tuesday, July 23, 2013 11:59 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: Deduplication/replication options Hi Sergio! Another thing to take into consideration: if you have switched from PVU licensing to sub-capacity licensing in the past: TSM sub-capacity licensing is based on the amount of data stored in your primary pool. If this data is stored on a de-duplicating storage device you will be charged for the gross amount of data. If you are using TSM de-duplication you will have to pay for the de-duplicated amount. This will probably save you a lot of money... Kind regards, Eric van Loon AF/KLM Storage Engineering -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Sergio O. Fuentes Sent: dinsdag 23 juli 2013 19:20 To: ADSM-L@VM.MARIST.EDU Subject: Deduplication/replication options Hello all, We're currently faced with a decision go with a dedupe storage array or with TSM dedupe for our backup storage targets. There are some very critical pros and cons going with one or the other. For example, TSM dedupe will reduce overall network throughput both for backups and replication (source-side dedupe would be used). A dedupe storage array won't do that for backup, but it would be possible if we replicated to an identical array (but TSM replication would be bandwidth intensive). TSM dedupe might not scale as well and may neccessitate more TSM servers to distribute the load. Overall, though, I think the cost of additional servers is way less than what a native dedupe array would cost so I don't think that's a big hit. Replication is key. We have two datacenters where I would love it if TSM replication could be used in order to quickly (still manually, though) activate the replication server for production if necessary. Having a dedupe storage array kind of removes that option, unless we want to replicate the whole rehydrated backup data via TSM. I'm going on and on here, but has anybody had to make a decision to go one way or the other? Would it make sense to do a hybrid deployment (combination of TSM Dedupe and Array dedupe)? Any thoughts or tales of woes and forewarnings are appreciated. Thanks! Sergio For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 33014286
Re: Deduplication/replication options
On 07/23/2013 06:30 PM, Nick Laflamme wrote: > I'm surprised by Allen's comments, given the context of the list. > > TSM doesn't support BOOST. It doesn't support at the server level, and it > doesn't support for a client writing directly to a DataDomain DDR. Duh, yes, good point. Context: We moved our Oracle backups off of TSM entirely. > On Tue, Jul 23, 2013 at 3:12 PM, Allen S. Rout wrote: >> We're not using boost; our primary use for the DD is for Oracle >> backups, and our DBAs are far more interested in the conventional >> filesystem user interface than they are in the network savings. So, in order to use the normal filesystem interface of the DD, they of course are not using TSM as an interlocutor. - Allen S. Rout
Re: Deduplication/replication options
Hi Sergio! Another thing to take into consideration: if you have switched from PVU licensing to sub-capacity licensing in the past: TSM sub-capacity licensing is based on the amount of data stored in your primary pool. If this data is stored on a de-duplicating storage device you will be charged for the gross amount of data. If you are using TSM de-duplication you will have to pay for the de-duplicated amount. This will probably save you a lot of money... Kind regards, Eric van Loon AF/KLM Storage Engineering -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Sergio O. Fuentes Sent: dinsdag 23 juli 2013 19:20 To: ADSM-L@VM.MARIST.EDU Subject: Deduplication/replication options Hello all, We're currently faced with a decision go with a dedupe storage array or with TSM dedupe for our backup storage targets. There are some very critical pros and cons going with one or the other. For example, TSM dedupe will reduce overall network throughput both for backups and replication (source-side dedupe would be used). A dedupe storage array won't do that for backup, but it would be possible if we replicated to an identical array (but TSM replication would be bandwidth intensive). TSM dedupe might not scale as well and may neccessitate more TSM servers to distribute the load. Overall, though, I think the cost of additional servers is way less than what a native dedupe array would cost so I don't think that's a big hit. Replication is key. We have two datacenters where I would love it if TSM replication could be used in order to quickly (still manually, though) activate the replication server for production if necessary. Having a dedupe storage array kind of removes that option, unless we want to replicate the whole rehydrated backup data via TSM. I'm going on and on here, but has anybody had to make a decision to go one way or the other? Would it make sense to do a hybrid deployment (combination of TSM Dedupe and Array dedupe)? Any thoughts or tales of woes and forewarnings are appreciated. Thanks! Sergio For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 33014286
Re: Deduplication/replication options
Thanks, guys, for your input. Nick, your comment is relevant to us. We're not used to by-passing TSM for any storage management task regarding backups. We use very little storage-based replication in our environment as it is, and introducing array-based replication adds a wrinkle to managing our backup retention and storage policies. Most of our application managers have decided to use application-based replication or clustering (dataguard for Oracle, Always-on for MSSQL, DAGS for Exchange, etc.). Stands to reason that we would try and use TSM replication for backups. I do like the idea of trying to squeeze out more disk space by compressing or deduplicating TSM deduplication extents. FYI, we had our business partner try this with compressing TSM deduplication extents on a V7000 array. According to them, IBM has tried this as well. The result is not sufficient to cover the cost of the V7000 compression license. (20% compression on average of TSM deduplication extents). So, IBM has said it's not a recommended practice for the V7000 array. I even had a DD POC sitting on the floor for some time and I didn't think to try it on TSM dedupe extents. Out of curiosity, has anyone experimented with ZFS compression on a TSM storage pool? Might be a low-cost option, but I'm not sure how scalable or stable it is. The options are numerous, our man-power is limited. Thanks for your help! SF On 7/23/13 6:30 PM, "Nick Laflamme" wrote: >I'm surprised by Allen's comments, given the context of the list. > >TSM doesn't support BOOST. It doesn't support at the server level, and it >doesn't support for a client writing directly to a DataDomain DDR. This >may >be obvious to everyone, but I fear for the people who are TSM-centric and >haven't gotten to the point of bypassing TSM in some instances. > >Also, BOOST is feature with a price, just like the VTL support is and >replication is. I'm not saying that's bad, only that you have to factor >that in. > >As for us, we don't use copy storage pools with our DataDomains; we make >sure our TSM servers use replicated disk and write to replicated storage, >and if we ever lose our primary data center, we'll restore at our DR site >pick up with the replicated DDR storage. As others have noted, this leaves >us vulnerable to any corruption due to DDR crashes or server crashes that >confuse the DDR, but management signed off on that. We briefly >experimented >with running copy pools on the same DDR to have diversity in how data was >arranged, but the growth in the size of our TSM databases and a >surprisingly poor dedupe rate for a second copy on the same DDR doomed >that >initiative. > >Nick
Re: Deduplication/replication options
I'm surprised by Allen's comments, given the context of the list. TSM doesn't support BOOST. It doesn't support at the server level, and it doesn't support for a client writing directly to a DataDomain DDR. This may be obvious to everyone, but I fear for the people who are TSM-centric and haven't gotten to the point of bypassing TSM in some instances. Also, BOOST is feature with a price, just like the VTL support is and replication is. I'm not saying that's bad, only that you have to factor that in. As for us, we don't use copy storage pools with our DataDomains; we make sure our TSM servers use replicated disk and write to replicated storage, and if we ever lose our primary data center, we'll restore at our DR site pick up with the replicated DDR storage. As others have noted, this leaves us vulnerable to any corruption due to DDR crashes or server crashes that confuse the DDR, but management signed off on that. We briefly experimented with running copy pools on the same DDR to have diversity in how data was arranged, but the growth in the size of our TSM databases and a surprisingly poor dedupe rate for a second copy on the same DDR doomed that initiative. Nick On Tue, Jul 23, 2013 at 3:12 PM, Allen S. Rout wrote: > On 07/23/2013 01:19 PM, Sergio O. Fuentes wrote: > > > > We're currently faced with a decision go with a dedupe storage array > > or with TSM dedupe for our backup storage targets. There are some > > very critical pros and cons going with one or the other. For > > example, TSM dedupe will reduce overall network throughput both for > > backups and replication (source-side dedupe would be used). A dedupe > > storage array won't do that for backup, > > > Not so. There's a driver-ish package from EMC, associated with the > Data Domain product line, called "boost". Boost shoves dedupe work > from the central device out to the client box, distributing CPU work > and saving network traffic. There may be other similar offerings, > but Data Domain is what we've got, so it's what I know. > > We're not using boost; our primary use for the DD is for Oracle > backups, and our DBAs are far more interested in the conventional > filesystem user interface than they are in the network savings. But > if you find the bandwidth between client and device to be a serious > bottleneck, there's an option. > > > > Replication is key. We have two datacenters where I would love it if > > TSM replication could be used in order to quickly (still manually, > > though) activate the replication server for production if necessary. > > Having a dedupe storage array kind of removes that option, unless we > > want to replicate the whole rehydrated backup data via TSM. > > I intend to go the same direction you are intending to go. But I'm > not there yet. I hope to have some results on this before September. > > > > Would it make sense to do a hybrid deployment (combination of TSM > > Dedupe and Array dedupe)? Any thoughts or tales of woes and > > forewarnings are appreciated. > > Only thoughts, not tales yet. But I'm planning to experiment with > dedupe both at the TSM level and at the storage array level. I've > heard several rumors that the Data Domain can dedupe even deduped > e.g. VEEAM backups, with very good ratios. I'm going to try a > similar theory with the DD and TSM-deduped stgpools. > > > - Allen S. Rout >
Re: Deduplication/replication options
I have no experience with TSM de-dup, but I have plenty with Data Domain. We have 3 different disaster recovery methods for 3 sites. 1. The largest site is traditional TSM, write the data to a primary pool (DD VTL) and make copies to physical tape and use a truck to move them away. 2. Medium site has a secondary DC 60 miles away and plenty of bandwidth. That site does DD to DD replication and disk array replication of the TSM disks. This results in a DR time of about 1 hour to start up the DR location. No copy tape. 3. A small site with DD to DD replication and a cold TSM instance at the DR site that will restore from the replicated DD. No copy tape. #1 is the safest because it uses copy tape. #2 & #3 get the data to safety the fastest. My preference is DD to DD replication and to make physical copy tapes of the production stuff. We have had corrupt tapes in the DD dues to the DD crashing. This is rare, but does happen. Both are valid depending on the amount of money you have to spend. We did not look at TSM de-dup because our servers are out of cycles already. Be sure to test any DR plans if you are using DD VTL. There are some interesting problems with serial number changes. Andy Huebner -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Sergio O. Fuentes Sent: Tuesday, July 23, 2013 12:20 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Deduplication/replication options Hello all, We're currently faced with a decision go with a dedupe storage array or with TSM dedupe for our backup storage targets. There are some very critical pros and cons going with one or the other. For example, TSM dedupe will reduce overall network throughput both for backups and replication (source-side dedupe would be used). A dedupe storage array won't do that for backup, but it would be possible if we replicated to an identical array (but TSM replication would be bandwidth intensive). TSM dedupe might not scale as well and may neccessitate more TSM servers to distribute the load. Overall, though, I think the cost of additional servers is way less than what a native dedupe array would cost so I don't think that's a big hit. Replication is key. We have two datacenters where I would love it if TSM replication could be used in order to quickly (still manually, though) activate the replication server for production if necessary. Having a dedupe storage array kind of removes that option, unless we want to replicate the whole rehydrated backup data via TSM. I'm going on and on here, but has anybody had to make a decision to go one way or the other? Would it make sense to do a hybrid deployment (combination of TSM Dedupe and Array dedupe)? Any thoughts or tales of woes and forewarnings are appreciated. Thanks! Sergio
Re: Deduplication/replication options
I'm using Data Domain as the only dedup component. Mgmt is balking at the cost additional disk or tape pools with TSM dedup and the highly desired "backup to non-dedup pool." Our current tape technology is quite old and replacing with several new drives and library hardware isn't on the financial agenda. We have Data Domain in two data centers. A TSM pool on the DD is replicated to the alternate DD via DD replication. It replicates the de-duped data, so latency/bandwidth is less of an issue. An second TSM server will see the other DD as a Pool, or at least that's the plan. Haven't fully tested yet. Had to carefully define the Device Class to make sure the path name is identical on both ends. Will have to stop DD replication, at least temporarily, to test it. But, haven't tested yet. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Sergio O. Fuentes Sent: Tuesday, July 23, 2013 12:20 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Deduplication/replication options Hello all, We're currently faced with a decision go with a dedupe storage array or with TSM dedupe for our backup storage targets. There are some very critical pros and cons going with one or the other. For example, TSM dedupe will reduce overall network throughput both for backups and replication (source-side dedupe would be used). A dedupe storage array won't do that for backup, but it would be possible if we replicated to an identical array (but TSM replication would be bandwidth intensive). TSM dedupe might not scale as well and may neccessitate more TSM servers to distribute the load. Overall, though, I think the cost of additional servers is way less than what a native dedupe array would cost so I don't think that's a big hit. Replication is key. We have two datacenters where I would love it if TSM replication could be used in order to quickly (still manually, though) activate the replication server for production if necessary. Having a dedupe storage array kind of removes that option, unless we want to replicate the whole rehydrated backup data via TSM. I'm going on and on here, but has anybody had to make a decision to go one way or the other? Would it make sense to do a hybrid deployment (combination of TSM Dedupe and Array dedupe)? Any thoughts or tales of woes and forewarnings are appreciated. Thanks! Sergio [Confidentiality notice:] *** This e-mail message, including attachments, if any, is intended for the person or entity to which it is addressed and may contain confidential or privileged information. Any unauthorized review, use, or disclosure is prohibited. If you are not the intended recipient, please contact the sender and destroy the original message, including all copies, Thank you. ***
Re: Deduplication/replication options
On 07/23/2013 01:19 PM, Sergio O. Fuentes wrote: > > We're currently faced with a decision go with a dedupe storage array > or with TSM dedupe for our backup storage targets. There are some > very critical pros and cons going with one or the other. For > example, TSM dedupe will reduce overall network throughput both for > backups and replication (source-side dedupe would be used). A dedupe > storage array won't do that for backup, Not so. There's a driver-ish package from EMC, associated with the Data Domain product line, called "boost". Boost shoves dedupe work from the central device out to the client box, distributing CPU work and saving network traffic. There may be other similar offerings, but Data Domain is what we've got, so it's what I know. We're not using boost; our primary use for the DD is for Oracle backups, and our DBAs are far more interested in the conventional filesystem user interface than they are in the network savings. But if you find the bandwidth between client and device to be a serious bottleneck, there's an option. > Replication is key. We have two datacenters where I would love it if > TSM replication could be used in order to quickly (still manually, > though) activate the replication server for production if necessary. > Having a dedupe storage array kind of removes that option, unless we > want to replicate the whole rehydrated backup data via TSM. I intend to go the same direction you are intending to go. But I'm not there yet. I hope to have some results on this before September. > Would it make sense to do a hybrid deployment (combination of TSM > Dedupe and Array dedupe)? Any thoughts or tales of woes and > forewarnings are appreciated. Only thoughts, not tales yet. But I'm planning to experiment with dedupe both at the TSM level and at the storage array level. I've heard several rumors that the Data Domain can dedupe even deduped e.g. VEEAM backups, with very good ratios. I'm going to try a similar theory with the DD and TSM-deduped stgpools. - Allen S. Rout
Re: Deduplication/replication options
Hi Sergio, There are many people more knowledgeable than I am on this topic, and I hope they contribute to this interesting question. My two cents would be to remember that the TSM database doesn't know about an array replication, so you'll have to deal with that issue if you have a massive recovery event. My other two cents are that to be certain sure that whichever solution you choose works as described. You can't test the alternatives enough. Best wishes, Keith Arbogast Indiana University
Deduplication/replication options
Hello all, We're currently faced with a decision go with a dedupe storage array or with TSM dedupe for our backup storage targets. There are some very critical pros and cons going with one or the other. For example, TSM dedupe will reduce overall network throughput both for backups and replication (source-side dedupe would be used). A dedupe storage array won't do that for backup, but it would be possible if we replicated to an identical array (but TSM replication would be bandwidth intensive). TSM dedupe might not scale as well and may neccessitate more TSM servers to distribute the load. Overall, though, I think the cost of additional servers is way less than what a native dedupe array would cost so I don't think that's a big hit. Replication is key. We have two datacenters where I would love it if TSM replication could be used in order to quickly (still manually, though) activate the replication server for production if necessary. Having a dedupe storage array kind of removes that option, unless we want to replicate the whole rehydrated backup data via TSM. I'm going on and on here, but has anybody had to make a decision to go one way or the other? Would it make sense to do a hybrid deployment (combination of TSM Dedupe and Array dedupe)? Any thoughts or tales of woes and forewarnings are appreciated. Thanks! Sergio
