V3700 for primary pools here, with AIX LVM mirroring between 2 datacenters. DB is residing on SVC auto-tiering on V7000 with a mix of SSD and 10K SAS drives, with SVC Enhanced MetroMirror. LTO4 on TS3310 for secondary (looking at replacing with a LTO7 library).
Yeah thanks to David for leading and bleeding; you've hopefully cleared the way for us mere followers. We've just implemented 7.1.6.000 and will start using Directory Container Pools soon. We're eager to migrate out of legacy dedup, as our single DB is now close to 1.3TB, and is hogged with quasi-constant runstats of the 2 large BF tables; even a few hours right after offline index rebuild. Regards, Luc -----Message d'origine----- De : ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] De la part de David Ehresman Envoyé : 24 août 2016 08:42 À : ADSM-L@VM.MARIST.EDU Objet : Re: [ADSM-L] *EXTERNAL* Re: deleing data from a containerpool We have an IBM v7000 as container storage on our local system and an IBM v5000 as container storage on our remote (DR) system. I agree with the David Nixon's comment about the number of TSM server upgrades we've had to go thru. We are about to upgrade to 7.1.6 to address another bug that is affecting our system. David Ehresman -----Original Message----- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Rhodes, Richard L. Sent: Wednesday, August 24, 2016 7:08 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] *EXTERNAL* Re: deleing data from a containerpool I'm curious, what kind of storage system do ya'll use for your container pools? -----Original Message----- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of David Ehresman Sent: Tuesday, August 23, 2016 8:56 AM To: ADSM-L@VM.MARIST.EDU Subject: *EXTERNAL* Re: deleing data from a containerpool We are getting a 78% savings on a mixed workload (BA, Oracle, VM, Exchange, SQL) with about 350 nodes moving around 10TB of data a night. David -----Original Message----- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Stefan Folkerts Sent: Tuesday, August 23, 2016 8:43 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] deleing data from a containerpool This inline deduplication and compression thing really is amazing; excerpt from q stgpool f=d Deduplication Savings: 153,840 G (90.83%) Compression Savings: 8,711 G (56.07%) Total Space Saved: 162,550 G (95.97%) So on top of the 90.83% deduplication saving (that dont' require any reclaiming) this customer saves another 56.07% of the stored data due to compression making it a total of 95.97% space saved. These figures require a lot of duplicate data of course but still, containerpools FTW! We call it the "cool pool" btw. :-) On Mon, Aug 22, 2016 at 4:01 PM, Del Hoobler <hoob...@us.ibm.com> wrote: > Minor correction. > > Inline compression for container pools was added in March in 7.1.5. > > > IBM Spectrum Protect 7.1.5 - Inline compression: > - Performed in-line after deduplication to provide additional storage > savings > - Negligible impact on resources – uses latest and most efficient > compression algorithms > - Can potentially double (or more) your storage savings after > deduplication > > > Del > > ---------------------------------------------------- > > > "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> wrote on 08/22/2016 > 09:53:08 AM: > > > From: "Loon, Eric van (ITOPT3) - KLM" <eric-van.l...@klm.com> > > To: ADSM-L@VM.MARIST.EDU > > Date: 08/22/2016 09:53 AM > > Subject: Re: deleing data from a containerpool Sent by: "ADSM: Dist > > Stor Manager" <ADSM-L@VM.MARIST.EDU> > > > > Indeed, TSM 7.1.0 to 7.1.5 only supported deduplication, additional > > compression was introduced in 7.1.6. > > Kind regards, > > Eric van Loon > > Air France/KLM Storage Engineering > > > > -----Original Message----- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of David Ehresman > > Sent: maandag 22 augustus 2016 15:12 > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: deleing data from a containerpool > > > > At the most recent levels of TSM, it both dedups and compresses but > > make sure you are at a level that does both. There was a level that > > only did dedup but not compression. > > > > David > > > > -----Original Message----- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of Rhodes, Richard L. > > Sent: Monday, August 22, 2016 9:07 AM > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: [ADSM-L] deleing data from a containerpool > > > > >But I totally agree, everyone who is using file device > > > > >classes or expensive backend deduplication (like Data Domain or > Protectier) > > > > >should seriously consider switching to container pools. > > > > > > > > > > > > We currently use DataDomains. > > > > With a DD it dedups what it can, then compresses the rest. > > > > Does TSM also try and compress what is leftover after dedup? > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of Loon, Eric van (ITOPT3) - KLM > > > > Sent: Tuesday, August 16, 2016 3:39 AM > > > > To: ADSM-L@VM.MARIST.EDU > > > > Subject: Re: deleing data from a containerpool > > > > > > > > Hi Stefan! > > > > Our database is on SSD in an IBM V3700, but the time needed for a > > del filespace can be significant though. But I totally agree, > > everyone who is using file device classes or expensive backend > > deduplication (like Data Domain or Protectier) should seriously > > consider switching to container pools. We are working on a design > > for our next TSM servers and we are able to lower our costs per TB > > by 75% compared to the old design based on the Data Domain! > > > > Kind regards, > > > > Eric van Loon > > > > Air France/KLM Storage Engineering > > > > > > > > -----Original Message----- > > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of Stefan Folkerts > > > > Sent: dinsdag 16 augustus 2016 8:33 > > > > To: ADSM-L@VM.MARIST.EDU > > > > Subject: Re: deleing data from a containerpool > > > > > > > > Yes, I too have noticed this and it is something to keep in mind. > > > > At the same time, I think almost everybody using this pool will be > > using SSD's for the database the impact will be overseeable. > > > > But the directory containerpool is still the best thing to happen to > > Spectrum Protect since replication came along if you ask me. great > > performance increase over fileclass restores, no more stopping > > reclaims during the day to increase restore performance, no more > > messing with numopenvolsallowed and reclaim values and number of > > processes to optimize daily operations and restore speed...oh, and > > compression that saves an easy 30-50% storage and license cost on > > top of the deduplication! > > > > > > > > > > > > > > > > > > > > On Mon, Aug 15, 2016 at 11:20 AM, Loon, Eric van (ITOPT3) - KLM < > > eric-van.l...@klm.com> wrote: > > > > > > > > > Hi all! > > > > > After doing some extensive testing with a directory container > > > > > storagepool I noticed a significant change compared to the old > > > > > traditional storage pools. > > > > > In a traditional storage pool TSM stores a file like an object. In > > > > > most cases one file is one object as far as I could see. Deleting > > > this > > > > > > data is very fast: a delete filespace runs rather fast because TSM > > > > > only has to delete the objects. So deleting a large database > > > client > > > > > with multiple TBs takes a few seconds or maybe a few minutes. > > > > > When you are using a container storage pool everything changes. > > > Files > > > > > are still stored as objects, but objects are split into chunks. > > > The > > > > > average size of a chuck is approx. 100 KB and TSM performs dedup > > > on > > > > > this chuck level. So if you now delete a large file, TSM has to > > > > > inspect every chunk to see if it is unique or not. When it is > > > unique > > > > > it will be deleted, otherwise not. If you delete a file which is > > > for > > > > > instance 40 GB in size, TSM has to inspect around 420,000 chucks > > > > > before the object can be deleted. I noticed that this takes > > > several > > > > > seconds to complete, so one has to take into consideration that > > > the > > > > > deletion of large clients requires significant more time to > > complete than one is used to. > > > > > Deleing a client with little more than 1 TB of Oracle data was > > > running > > > > > > for more than 20 minutes. So a delete filespace for really large > > > > > database clients can run for hours! Has anyone else noticed this > > behavior too? > > > > > Kind regards, > > > > > Eric van Loon > > > > > Air France/KLM Storage Engineering > > > > > ******************************************************** > > > > > For information, services and offers, please visit our web site: > > > > > https://urldefense.proofpoint.com/v2/url? > > > u=http-3A__www.klm.com&d=AwIGaQ&c=SgMrq23dbjbGX6e0ZsSHgEZX6A4IAf > 1SO3AJ2bNrHlk&r=dOGCMY197NTNH1k_wcsrWS3_fxedKW4rpKJ8cHCD2L8&m= > vUuUnchIk8qp8ANX9ecD5HSZje8iCRgiNUPhmahQWTQ&s= > eE7XROkp9Iv02y6CJDM84muZgTbKNhpt7nAgYPrCs-0&e= > > . This e-mail and any attachment may contain > > > > > confidential and privileged material intended for the addressee only. > > > > > If you are not the addressee, you are notified that no part of the > > > > > e-mail or any attachment may be disclosed, copied or distributed, > > > and > > > > > that any other action related to this e-mail or attachment is > > > strictly > > > > > > prohibited, and may be unlawful. If you have received this e-mail > > > by > > > > > error, please notify the sender immediately by return e-mail, and > > delete this message. > > > > > > > > > > Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries > > > and/or > > > > > its employees shall not be liable for the incorrect or incomplete > > > > > transmission of this e-mail or any attachments, nor responsible > > for any delay in receipt. > > > > > Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal > > > > > Dutch > > > > > Airlines) is registered in Amstelveen, The Netherlands, with > > > > > registered number 33014286 > > > > > ******************************************************** > > > > > > > > > ******************************************************** > > > > For information, services and offers, please visit our web site: > > https://urldefense.proofpoint.com/v2/url? > > > u=http-3A__www.klm.com&d=AwIGaQ&c=SgMrq23dbjbGX6e0ZsSHgEZX6A4IAf > 1SO3AJ2bNrHlk&r=dOGCMY197NTNH1k_wcsrWS3_fxedKW4rpKJ8cHCD2L8&m= > vUuUnchIk8qp8ANX9ecD5HSZje8iCRgiNUPhmahQWTQ&s= > eE7XROkp9Iv02y6CJDM84muZgTbKNhpt7nAgYPrCs-0&e= > > . This e-mail and any attachment may contain confidential and > > privileged material intended for the addressee only. If you are not > > the addressee, you are notified that no part of the e-mail or any > > attachment may be disclosed, copied or distributed, and that any > > other action related to this e-mail or attachment is strictly > > prohibited, and may be unlawful. If you have received this e-mail by > > error, please notify the sender immediately by return e-mail, and > > delete this message. > > > > > > > > Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/ > > or its employees shall not be liable for the incorrect or incomplete > > transmission of this e-mail or any attachments, nor responsible for > > any delay in receipt. > > > > Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal > > Dutch Airlines) is registered in Amstelveen, The Netherlands, with > > registered number 33014286 > > > > ******************************************************** > > > > > > > > ******************************************************** > > For information, services and offers, please visit our web site: > > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.klm.com&d=Aw > > IFaQ&c=SgMrq23dbjbGX6e0ZsSHgEZX6A4IAf1SO3AJ2bNrHlk&r=dOGCMY197NTNH1k > > _wcsrWS3_fxedKW4rpKJ8cHCD2L8&m=2khjm5kIUk4HAMhbAmSErW4psymX49EvvcPlo > > nm458c&s=KERJhiJfWrogTNbgmZOr6_HGAgfaXIQntXRwL231Di4&e= . This > > e-mail and any attachment may contain confidential and privileged > > material intended for the addressee only. If you are not the > > addressee, you are notified that no part of the e-mail or any > > attachment may be disclosed, copied or distributed, and that any other > > action related to this e-mail or attachment is strictly prohibited, and may > > be unlawful. If you have received this e-mail by error, please notify the > > sender immediately by return e-mail, and delete this message. > > > > Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/ > > or its employees shall not be liable for the incorrect or incomplete > > transmission of this e-mail or any attachments, nor responsible for > > any delay in receipt. > > Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal > > Dutch Airlines) is registered in Amstelveen, The Netherlands, with > > registered number 33014286 > > ******************************************************** > > > > >
