Re: *EXTERNAL* Re: deleing data from a containerpool
We often use IBM Storwize systems, V3700 for smaller setups and V5020 for larger configurations. >From 24 up to 60 4-6TB nearline drives all in RAID 6. 2x8 core Lenovo servers with internal SSD's in RAID 5 attached to (super fast) internal raid controllers for database and active log reaching 140K iop/s in IBM Spectrum Protect database benchmarks. We outfit them with 192GB of memory when using replication. These systems do 800MB/s inline deduplication and compression without CPU room to spare in some load testing. I have seen that 10Gb/s line filled up many times and even use LACP configurations with two 10Gb/s adapters to make sure I'm not holding it back. We usually run Linux, SLES to be exact. The first time I saw one of these things do inline dedup at those speeds...the inner nerd got a little emotional. :-) We've really got some amazing tech right here with these containerpools. On Wed, Aug 24, 2016 at 1:08 PM, Rhodes, Richard L. < rrho...@firstenergycorp.com> wrote: > I'm curious, what kind of storage system do ya'll use for your container > pools? > > > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of > David Ehresman > Sent: Tuesday, August 23, 2016 8:56 AM > To: ADSM-L@VM.MARIST.EDU > Subject: *EXTERNAL* Re: deleing data from a containerpool > > We are getting a 78% savings on a mixed workload (BA, Oracle, VM, > Exchange, SQL) with about 350 nodes moving around 10TB of data a night. > > David > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of > Stefan Folkerts > Sent: Tuesday, August 23, 2016 8:43 AM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: [ADSM-L] deleing data from a containerpool > > This inline deduplication and compression thing really is amazing; > > excerpt from q stgpool f=d > > Deduplication Savings: 153,840 G (90.83%) > Compression Savings: 8,711 G (56.07%) > Total Space Saved: 162,550 G (95.97%) > > So on top of the 90.83% deduplication saving (that dont' require any > reclaiming) this customer saves another 56.07% of the stored data due to > compression making it a total of 95.97% space saved. > These figures require a lot of duplicate data of course but still, > containerpools FTW! > We call it the "cool pool" btw. :-) > > > > On Mon, Aug 22, 2016 at 4:01 PM, Del Hoobler <hoob...@us.ibm.com> wrote: > > > Minor correction. > > > > Inline compression for container pools was added in March in 7.1.5. > > > > > > IBM Spectrum Protect 7.1.5 - Inline compression: > > - Performed in-line after deduplication to provide additional storage > > savings > > - Negligible impact on resources – uses latest and most efficient > > compression algorithms > > - Can potentially double (or more) your storage savings after > > deduplication > > > > > > Del > > > > > > > > > > "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> wrote on 08/22/2016 > > 09:53:08 AM: > > > > > From: "Loon, Eric van (ITOPT3) - KLM" <eric-van.l...@klm.com> > > > To: ADSM-L@VM.MARIST.EDU > > > Date: 08/22/2016 09:53 AM > > > Subject: Re: deleing data from a containerpool > > > Sent by: "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> > > > > > > Indeed, TSM 7.1.0 to 7.1.5 only supported deduplication, additional > > > compression was introduced in 7.1.6. > > > Kind regards, > > > Eric van Loon > > > Air France/KLM Storage Engineering > > > > > > -Original Message- > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > > Behalf Of David Ehresman > > > Sent: maandag 22 augustus 2016 15:12 > > > To: ADSM-L@VM.MARIST.EDU > > > Subject: Re: deleing data from a containerpool > > > > > > At the most recent levels of TSM, it both dedups and compresses but > > > make sure you are at a level that does both. There was a level that > > > only did dedup but not compression. > > > > > > David > > > > > > -Original Message- > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > > Behalf Of Rhodes, Richard L. > > > Sent: Monday, August 22, 2016 9:07 AM > > > To: ADSM-L@VM.MARIST.EDU > > > Subject: Re: [ADSM-L] deleing data from a containerpool > > > > > > >But I totally agree, everyone who is using file device > > > > > > >classes o
Re: *EXTERNAL* Re: deleing data from a containerpool
Charles, I am now dealing with the same issue you mentioned in bullet item 2 and have with an active PMR with IBM. The container storage pool replicate and protect processes are failing with (ANR0522W) messages claiming the container storage pool is out of space but a quick query reports that it is roughly 38% utilized. We have been able to identify that the free space in the container pool is allocated for "dedup data" and it is the space allocated for "non-dedup data" is full and ultimately causing the errors. The space allocated/available for these data types can be queried by issuing the show sdpool command from the server administrative command line interface. Now I would think that there would be a mechanism by which the storage pool would just re-allocate some of the available free space to the storage or the non-dedup data and keep right on working, but apparently that is not the case, or is not working as designed. I am continuing to work with support on the issue and will update this thread when it is resolved/closed. Unfortunately, for my problem upgrading to version 7.1.6 does not resolve the problem. -Rick Adamson -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Nixon, Charles D. (David) Sent: Wednesday, August 24, 2016 8:32 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] *EXTERNAL* Re: deleing data from a containerpool We have a HDS G400 behind our new TSM servers FMD/Flash behind one container for our primary production database (11.5 TB with 16 versions is fitting in 16.6TB) and the rest is in a giant nearline SAS pool. The flash also serves to back the logs/db and the SAS serves as a DBB/logarch location. Other lessons learned from container pools... 1. Make sure that your TSM server is big enough. We have had to upgrade some old Power 750's to new boxes to handle the overhead of compression and dedupe. This offsets some of the finical savings from getting rid of purpose built appliances. 2. In EVERY SINGLE VERSION that we have tried from 7.1.3.200, 7.1.4.x, to 7.1.5.x we have run into container related bugs. As such, we keep having to upgrade the TSM server, only to find new bugs introduced in the newer versions. TSM used to be a set and forget application for us. Now we are stuck in a horrible upgrade cycle for the last year and it doesn't seem to be letting up. Last week we were told that we may need to go to 7.1.6 since TSM thinks a container is full when it's not. --- David Nixon Storage Engineer II Technology Services Group Carilion Clinic 451 Kimball Ave. Roanoke, VA 24015 Phone: 540-224-3903 cdni...@carilionclinic.org Our mission: Improve the health of the communities we serve. From: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] on behalf of Rhodes, Richard L. [rrho...@firstenergycorp.com] Sent: Wednesday, August 24, 2016 7:08 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] *EXTERNAL* Re: deleing data from a containerpool I'm curious, what kind of storage system do ya'll use for your container pools? -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of David Ehresman Sent: Tuesday, August 23, 2016 8:56 AM To: ADSM-L@VM.MARIST.EDU Subject: *EXTERNAL* Re: deleing data from a containerpool We are getting a 78% savings on a mixed workload (BA, Oracle, VM, Exchange, SQL) with about 350 nodes moving around 10TB of data a night. David -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Stefan Folkerts Sent: Tuesday, August 23, 2016 8:43 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] deleing data from a containerpool This inline deduplication and compression thing really is amazing; excerpt from q stgpool f=d Deduplication Savings: 153,840 G (90.83%) Compression Savings: 8,711 G (56.07%) Total Space Saved: 162,550 G (95.97%) So on top of the 90.83% deduplication saving (that dont' require any reclaiming) this customer saves another 56.07% of the stored data due to compression making it a total of 95.97% space saved. These figures require a lot of duplicate data of course but still, containerpools FTW! We call it the "cool pool" btw. :-) On Mon, Aug 22, 2016 at 4:01 PM, Del Hoobler <hoob...@us.ibm.com> wrote: > Minor correction. > > Inline compression for container pools was added in March in 7.1.5. > > > IBM Spectrum Protect 7.1.5 - Inline compression: > - Performed in-line after deduplication to provide additional storage > savings > - Negligible impact on resources - uses latest and most efficient > compression algorithms > - Can potentially double (or more) your storage savings after > deduplication > > > Del > > ---
Re: *EXTERNAL* Re: deleing data from a containerpool
V3700 for primary pools here, with AIX LVM mirroring between 2 datacenters. DB is residing on SVC auto-tiering on V7000 with a mix of SSD and 10K SAS drives, with SVC Enhanced MetroMirror. LTO4 on TS3310 for secondary (looking at replacing with a LTO7 library). Yeah thanks to David for leading and bleeding; you've hopefully cleared the way for us mere followers. We've just implemented 7.1.6.000 and will start using Directory Container Pools soon. We're eager to migrate out of legacy dedup, as our single DB is now close to 1.3TB, and is hogged with quasi-constant runstats of the 2 large BF tables; even a few hours right after offline index rebuild. Regards, Luc -Message d'origine- De : ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] De la part de David Ehresman Envoyé : 24 août 2016 08:42 À : ADSM-L@VM.MARIST.EDU Objet : Re: [ADSM-L] *EXTERNAL* Re: deleing data from a containerpool We have an IBM v7000 as container storage on our local system and an IBM v5000 as container storage on our remote (DR) system. I agree with the David Nixon's comment about the number of TSM server upgrades we've had to go thru. We are about to upgrade to 7.1.6 to address another bug that is affecting our system. David Ehresman -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Rhodes, Richard L. Sent: Wednesday, August 24, 2016 7:08 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] *EXTERNAL* Re: deleing data from a containerpool I'm curious, what kind of storage system do ya'll use for your container pools? -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of David Ehresman Sent: Tuesday, August 23, 2016 8:56 AM To: ADSM-L@VM.MARIST.EDU Subject: *EXTERNAL* Re: deleing data from a containerpool We are getting a 78% savings on a mixed workload (BA, Oracle, VM, Exchange, SQL) with about 350 nodes moving around 10TB of data a night. David -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Stefan Folkerts Sent: Tuesday, August 23, 2016 8:43 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] deleing data from a containerpool This inline deduplication and compression thing really is amazing; excerpt from q stgpool f=d Deduplication Savings: 153,840 G (90.83%) Compression Savings: 8,711 G (56.07%) Total Space Saved: 162,550 G (95.97%) So on top of the 90.83% deduplication saving (that dont' require any reclaiming) this customer saves another 56.07% of the stored data due to compression making it a total of 95.97% space saved. These figures require a lot of duplicate data of course but still, containerpools FTW! We call it the "cool pool" btw. :-) On Mon, Aug 22, 2016 at 4:01 PM, Del Hoobler <hoob...@us.ibm.com> wrote: > Minor correction. > > Inline compression for container pools was added in March in 7.1.5. > > > IBM Spectrum Protect 7.1.5 - Inline compression: > - Performed in-line after deduplication to provide additional storage > savings > - Negligible impact on resources – uses latest and most efficient > compression algorithms > - Can potentially double (or more) your storage savings after > deduplication > > > Del > > > > > "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> wrote on 08/22/2016 > 09:53:08 AM: > > > From: "Loon, Eric van (ITOPT3) - KLM" <eric-van.l...@klm.com> > > To: ADSM-L@VM.MARIST.EDU > > Date: 08/22/2016 09:53 AM > > Subject: Re: deleing data from a containerpool Sent by: "ADSM: Dist > > Stor Manager" <ADSM-L@VM.MARIST.EDU> > > > > Indeed, TSM 7.1.0 to 7.1.5 only supported deduplication, additional > > compression was introduced in 7.1.6. > > Kind regards, > > Eric van Loon > > Air France/KLM Storage Engineering > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of David Ehresman > > Sent: maandag 22 augustus 2016 15:12 > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: deleing data from a containerpool > > > > At the most recent levels of TSM, it both dedups and compresses but > > make sure you are at a level that does both. There was a level that > > only did dedup but not compression. > > > > David > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of Rhodes, Richard L. > > Sent: Monday, August 22, 2016 9:07 AM > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: [ADSM-L] deleing data from a containerpool > > > > >But I totally agree, everyone who is usi
Re: *EXTERNAL* Re: deleing data from a containerpool
We have an IBM v7000 as container storage on our local system and an IBM v5000 as container storage on our remote (DR) system. I agree with the David Nixon's comment about the number of TSM server upgrades we've had to go thru. We are about to upgrade to 7.1.6 to address another bug that is affecting our system. David Ehresman -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Rhodes, Richard L. Sent: Wednesday, August 24, 2016 7:08 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] *EXTERNAL* Re: deleing data from a containerpool I'm curious, what kind of storage system do ya'll use for your container pools? -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of David Ehresman Sent: Tuesday, August 23, 2016 8:56 AM To: ADSM-L@VM.MARIST.EDU Subject: *EXTERNAL* Re: deleing data from a containerpool We are getting a 78% savings on a mixed workload (BA, Oracle, VM, Exchange, SQL) with about 350 nodes moving around 10TB of data a night. David -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Stefan Folkerts Sent: Tuesday, August 23, 2016 8:43 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] deleing data from a containerpool This inline deduplication and compression thing really is amazing; excerpt from q stgpool f=d Deduplication Savings: 153,840 G (90.83%) Compression Savings: 8,711 G (56.07%) Total Space Saved: 162,550 G (95.97%) So on top of the 90.83% deduplication saving (that dont' require any reclaiming) this customer saves another 56.07% of the stored data due to compression making it a total of 95.97% space saved. These figures require a lot of duplicate data of course but still, containerpools FTW! We call it the "cool pool" btw. :-) On Mon, Aug 22, 2016 at 4:01 PM, Del Hoobler <hoob...@us.ibm.com> wrote: > Minor correction. > > Inline compression for container pools was added in March in 7.1.5. > > > IBM Spectrum Protect 7.1.5 - Inline compression: > - Performed in-line after deduplication to provide additional storage > savings > - Negligible impact on resources – uses latest and most efficient > compression algorithms > - Can potentially double (or more) your storage savings after > deduplication > > > Del > > > > > "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> wrote on 08/22/2016 > 09:53:08 AM: > > > From: "Loon, Eric van (ITOPT3) - KLM" <eric-van.l...@klm.com> > > To: ADSM-L@VM.MARIST.EDU > > Date: 08/22/2016 09:53 AM > > Subject: Re: deleing data from a containerpool > > Sent by: "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> > > > > Indeed, TSM 7.1.0 to 7.1.5 only supported deduplication, additional > > compression was introduced in 7.1.6. > > Kind regards, > > Eric van Loon > > Air France/KLM Storage Engineering > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of David Ehresman > > Sent: maandag 22 augustus 2016 15:12 > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: deleing data from a containerpool > > > > At the most recent levels of TSM, it both dedups and compresses but > > make sure you are at a level that does both. There was a level that > > only did dedup but not compression. > > > > David > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of Rhodes, Richard L. > > Sent: Monday, August 22, 2016 9:07 AM > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: [ADSM-L] deleing data from a containerpool > > > > >But I totally agree, everyone who is using file device > > > > >classes or expensive backend deduplication (like Data Domain or > Protectier) > > > > >should seriously consider switching to container pools. > > > > > > > > > > > > We currently use DataDomains. > > > > With a DD it dedups what it can, then compresses the rest. > > > > Does TSM also try and compress what is leftover after dedup? > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Original Message- > > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of Loon, Eric van (ITOPT3) - KLM > > > > Sent: Tuesday, August 16, 2016 3:39 AM > > > > To: ADSM-L@VM.MARIST.EDU > > > > Subject: Re: deleing data from a containerpool > > > >
Re: *EXTERNAL* Re: deleing data from a containerpool
We have a HDS G400 behind our new TSM servers FMD/Flash behind one container for our primary production database (11.5 TB with 16 versions is fitting in 16.6TB) and the rest is in a giant nearline SAS pool. The flash also serves to back the logs/db and the SAS serves as a DBB/logarch location. Other lessons learned from container pools... 1. Make sure that your TSM server is big enough. We have had to upgrade some old Power 750's to new boxes to handle the overhead of compression and dedupe. This offsets some of the finical savings from getting rid of purpose built appliances. 2. In EVERY SINGLE VERSION that we have tried from 7.1.3.200, 7.1.4.x, to 7.1.5.x we have run into container related bugs. As such, we keep having to upgrade the TSM server, only to find new bugs introduced in the newer versions. TSM used to be a set and forget application for us. Now we are stuck in a horrible upgrade cycle for the last year and it doesn't seem to be letting up. Last week we were told that we may need to go to 7.1.6 since TSM thinks a container is full when it's not. --- David Nixon Storage Engineer II Technology Services Group Carilion Clinic 451 Kimball Ave. Roanoke, VA 24015 Phone: 540-224-3903 cdni...@carilionclinic.org Our mission: Improve the health of the communities we serve. From: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] on behalf of Rhodes, Richard L. [rrho...@firstenergycorp.com] Sent: Wednesday, August 24, 2016 7:08 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] *EXTERNAL* Re: deleing data from a containerpool I'm curious, what kind of storage system do ya'll use for your container pools? -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of David Ehresman Sent: Tuesday, August 23, 2016 8:56 AM To: ADSM-L@VM.MARIST.EDU Subject: *EXTERNAL* Re: deleing data from a containerpool We are getting a 78% savings on a mixed workload (BA, Oracle, VM, Exchange, SQL) with about 350 nodes moving around 10TB of data a night. David -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Stefan Folkerts Sent: Tuesday, August 23, 2016 8:43 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] deleing data from a containerpool This inline deduplication and compression thing really is amazing; excerpt from q stgpool f=d Deduplication Savings: 153,840 G (90.83%) Compression Savings: 8,711 G (56.07%) Total Space Saved: 162,550 G (95.97%) So on top of the 90.83% deduplication saving (that dont' require any reclaiming) this customer saves another 56.07% of the stored data due to compression making it a total of 95.97% space saved. These figures require a lot of duplicate data of course but still, containerpools FTW! We call it the "cool pool" btw. :-) On Mon, Aug 22, 2016 at 4:01 PM, Del Hoobler <hoob...@us.ibm.com> wrote: > Minor correction. > > Inline compression for container pools was added in March in 7.1.5. > > > IBM Spectrum Protect 7.1.5 - Inline compression: > - Performed in-line after deduplication to provide additional storage > savings > - Negligible impact on resources – uses latest and most efficient > compression algorithms > - Can potentially double (or more) your storage savings after > deduplication > > > Del > > > > > "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> wrote on 08/22/2016 > 09:53:08 AM: > > > From: "Loon, Eric van (ITOPT3) - KLM" <eric-van.l...@klm.com> > > To: ADSM-L@VM.MARIST.EDU > > Date: 08/22/2016 09:53 AM > > Subject: Re: deleing data from a containerpool > > Sent by: "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> > > > > Indeed, TSM 7.1.0 to 7.1.5 only supported deduplication, additional > > compression was introduced in 7.1.6. > > Kind regards, > > Eric van Loon > > Air France/KLM Storage Engineering > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of David Ehresman > > Sent: maandag 22 augustus 2016 15:12 > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: deleing data from a containerpool > > > > At the most recent levels of TSM, it both dedups and compresses but > > make sure you are at a level that does both. There was a level that > > only did dedup but not compression. > > > > David > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of Rhodes, Richard L. > > Sent: Monday, August 22, 2016 9:07 AM > > To: ADSM-L@VM.MAR
Re: *EXTERNAL* Re: deleing data from a containerpool
Hi Richard! We decided to go for one of the storage devices from the Spectrum Protect Blueprints: IBM Storwize V3700 for the engineering environment and we will use the larger V5010 for the future production environment. Kind regards, Eric van Loon Air France/KLM Storage Engineering -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Rhodes, Richard L. Sent: woensdag 24 augustus 2016 13:08 To: ADSM-L@VM.MARIST.EDU Subject: Re: *EXTERNAL* Re: deleing data from a containerpool I'm curious, what kind of storage system do ya'll use for your container pools? -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of David Ehresman Sent: Tuesday, August 23, 2016 8:56 AM To: ADSM-L@VM.MARIST.EDU Subject: *EXTERNAL* Re: deleing data from a containerpool We are getting a 78% savings on a mixed workload (BA, Oracle, VM, Exchange, SQL) with about 350 nodes moving around 10TB of data a night. David -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Stefan Folkerts Sent: Tuesday, August 23, 2016 8:43 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] deleing data from a containerpool This inline deduplication and compression thing really is amazing; excerpt from q stgpool f=d Deduplication Savings: 153,840 G (90.83%) Compression Savings: 8,711 G (56.07%) Total Space Saved: 162,550 G (95.97%) So on top of the 90.83% deduplication saving (that dont' require any reclaiming) this customer saves another 56.07% of the stored data due to compression making it a total of 95.97% space saved. These figures require a lot of duplicate data of course but still, containerpools FTW! We call it the "cool pool" btw. :-) On Mon, Aug 22, 2016 at 4:01 PM, Del Hoobler <hoob...@us.ibm.com> wrote: > Minor correction. > > Inline compression for container pools was added in March in 7.1.5. > > > IBM Spectrum Protect 7.1.5 - Inline compression: > - Performed in-line after deduplication to provide additional storage > savings > - Negligible impact on resources – uses latest and most efficient > compression algorithms > - Can potentially double (or more) your storage savings after > deduplication > > > Del > > > > > "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> wrote on 08/22/2016 > 09:53:08 AM: > > > From: "Loon, Eric van (ITOPT3) - KLM" <eric-van.l...@klm.com> > > To: ADSM-L@VM.MARIST.EDU > > Date: 08/22/2016 09:53 AM > > Subject: Re: deleing data from a containerpool Sent by: "ADSM: Dist > > Stor Manager" <ADSM-L@VM.MARIST.EDU> > > > > Indeed, TSM 7.1.0 to 7.1.5 only supported deduplication, additional > > compression was introduced in 7.1.6. > > Kind regards, > > Eric van Loon > > Air France/KLM Storage Engineering > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of David Ehresman > > Sent: maandag 22 augustus 2016 15:12 > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: deleing data from a containerpool > > > > At the most recent levels of TSM, it both dedups and compresses but > > make sure you are at a level that does both. There was a level that > > only did dedup but not compression. > > > > David > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of Rhodes, Richard L. > > Sent: Monday, August 22, 2016 9:07 AM > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: [ADSM-L] deleing data from a containerpool > > > > >But I totally agree, everyone who is using file device > > > > >classes or expensive backend deduplication (like Data Domain or > Protectier) > > > > >should seriously consider switching to container pools. > > > > > > > > > > > > We currently use DataDomains. > > > > With a DD it dedups what it can, then compresses the rest. > > > > Does TSM also try and compress what is leftover after dedup? > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Original Message- > > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of Loon, Eric van (ITOPT3) - KLM > > > > Sent: Tuesday, August 16, 2016 3:39 AM > > > > To: ADSM-L@VM.MARIST.EDU > > > > Subject: Re: deleing data from a containerpool > > > > > > > > Hi Stefan! > > > &
Re: *EXTERNAL* Re: deleing data from a containerpool
I'm curious, what kind of storage system do ya'll use for your container pools? -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of David Ehresman Sent: Tuesday, August 23, 2016 8:56 AM To: ADSM-L@VM.MARIST.EDU Subject: *EXTERNAL* Re: deleing data from a containerpool We are getting a 78% savings on a mixed workload (BA, Oracle, VM, Exchange, SQL) with about 350 nodes moving around 10TB of data a night. David -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Stefan Folkerts Sent: Tuesday, August 23, 2016 8:43 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] deleing data from a containerpool This inline deduplication and compression thing really is amazing; excerpt from q stgpool f=d Deduplication Savings: 153,840 G (90.83%) Compression Savings: 8,711 G (56.07%) Total Space Saved: 162,550 G (95.97%) So on top of the 90.83% deduplication saving (that dont' require any reclaiming) this customer saves another 56.07% of the stored data due to compression making it a total of 95.97% space saved. These figures require a lot of duplicate data of course but still, containerpools FTW! We call it the "cool pool" btw. :-) On Mon, Aug 22, 2016 at 4:01 PM, Del Hoobler <hoob...@us.ibm.com> wrote: > Minor correction. > > Inline compression for container pools was added in March in 7.1.5. > > > IBM Spectrum Protect 7.1.5 - Inline compression: > - Performed in-line after deduplication to provide additional storage > savings > - Negligible impact on resources – uses latest and most efficient > compression algorithms > - Can potentially double (or more) your storage savings after > deduplication > > > Del > > > > > "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> wrote on 08/22/2016 > 09:53:08 AM: > > > From: "Loon, Eric van (ITOPT3) - KLM" <eric-van.l...@klm.com> > > To: ADSM-L@VM.MARIST.EDU > > Date: 08/22/2016 09:53 AM > > Subject: Re: deleing data from a containerpool > > Sent by: "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> > > > > Indeed, TSM 7.1.0 to 7.1.5 only supported deduplication, additional > > compression was introduced in 7.1.6. > > Kind regards, > > Eric van Loon > > Air France/KLM Storage Engineering > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of David Ehresman > > Sent: maandag 22 augustus 2016 15:12 > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: deleing data from a containerpool > > > > At the most recent levels of TSM, it both dedups and compresses but > > make sure you are at a level that does both. There was a level that > > only did dedup but not compression. > > > > David > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of Rhodes, Richard L. > > Sent: Monday, August 22, 2016 9:07 AM > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: [ADSM-L] deleing data from a containerpool > > > > >But I totally agree, everyone who is using file device > > > > >classes or expensive backend deduplication (like Data Domain or > Protectier) > > > > >should seriously consider switching to container pools. > > > > > > > > > > > > We currently use DataDomains. > > > > With a DD it dedups what it can, then compresses the rest. > > > > Does TSM also try and compress what is leftover after dedup? > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Original Message- > > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On > > Behalf Of Loon, Eric van (ITOPT3) - KLM > > > > Sent: Tuesday, August 16, 2016 3:39 AM > > > > To: ADSM-L@VM.MARIST.EDU > > > > Subject: Re: deleing data from a containerpool > > > > > > > > Hi Stefan! > > > > Our database is on SSD in an IBM V3700, but the time needed for a > > del filespace can be significant though. But I totally agree, > > everyone who is using file device classes or expensive backend > > deduplication (like Data Domain or Protectier) should seriously > > consider switching to container pools. We are working on a design > > for our next TSM servers and we are able to lower our costs per TB > > by 75% compared to the old design based on the Data Domain! > > > > Kind
