Re: Antwort: Re: low bandwitdth and big files
Peter, If I'm running compression on all of my clients, why would I attempt to turn it on at the devclass level. The only thing I can see it doing is prevent me from streaming because it's first going to try and compress anything I send to. Hence, an overall performance hit. Regards, Joe -Original Message- From: Peter Sattler [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 31, 2002 12:17 PM To: [EMAIL PROTECTED] Subject: Antwort: Re: low bandwitdth and big files Hi Joe, I strongly agree with Matt. I've done then same thing, that is client side compression and hardware compression on tape. With TSM, with Networker - I've never seen problems. On the server side all you lose is a bit of tape capacity. The compression is hardware (sic!) not software. The well known advice is for software compression - and there it is necessary to avoid double compression because it takes away resources. So in your case try client side compression - it might well be that you gain more than a doubled data rate. Cheers Peter Wholey, Joseph (TGA\\MLOL) [EMAIL PROTECTED]@VM.MARIST.EDU am 30.01.2002 22:07:19 Bitte antworten an ADSM: Dist Stor Manager [EMAIL PROTECTED] Gesendet von: ADSM: Dist Stor Manager [EMAIL PROTECTED] An:[EMAIL PROTECTED] Kopie: Thema: Re: low bandwitdth and big files Matt, Don't be so sure... my MVS folks are telling me that there is no way TSM is over riding their micro-code hardware level compression. (but I will double check with them and again as well as with Tivoli). With regard to compressing data twice, I disagree. There's something very wrong with it. That's why it is strongly recommended not to do it. (not just with TSM, but with all data) Some data that goes thru multiple compression routines can blow up to 2x the size the file started out as. And finally, the reason we turn compression on at the client, is to compress it before it rides our very slow network. Otherwise, I wouldn't. Regards, Joe -Original Message- From: Matthew Glanville [mailto:[EMAIL PROTECTED]] Sent: Wednesday, January 30, 2002 3:42 PM To: [EMAIL PROTECTED] Subject: Re: low bandwitdth and big files From: Matthew Glanville Joe, I do not believe there is such a thing as 'server' level compression. My understanding is that the device class compression settings are reflecting the hardware level compression settings, they can override what the microcode may have set the 'default' to. We have no problems at all with clients that compress with the tsm client, and then compress again on the tape drive, you loose just a little bit of space, and yes, the occupancy information does not know that the data has already been compresssed. There is really nothing wrong with compressing data more than once, the files get a bit bigger, but it could be worth the time and bandwith saved. Also don't forget that lots of data is stored already compressed in zipped files or compressed images like jpeg and mpeg. I would not touch with the compression settings on the device class, keep them on at the highest level, just turn on or off the TSM client compression as needed. Check and see if that helps your low bandwith backups. Matthew Glanville [EMAIL PROTECTED] Wholey, Joseph (TGA\\MLOL) [EMAIL PROTECTED]@VM.MARIST.EDU on 01/30/2002 01:39:10 PM Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: Subject: Re: low bandwitdth and big files Matt, Are you running two, maybe 3 compression routines. i.e. once at the client, once at the server level (you'll see if you q devclass f=d on a sequential storage device) and once at the hardware level (micro code). If so, have you kept check on the amount of data in your stgpool. I say this because a q occ on a file space is not going to give you an accurrate indication of the amount of data that node/filespace has dumped into your stgpool. Although the IBMers and the manuals say don't run multiple compression routines, they've yet to advise on what to do if you have to run client side compression due to a slow network. I can shut off server side/devclass compression, but what about hardware compression. Can you shut off compression on a 3590 tape device. Isn't that a micro code issue. i.e. you can't shut it off? Regards, Joe -Original Message- From: Matthew Glanville [mailto:[EMAIL PROTECTED]] Sent: Wednesday, January 30, 2002 12:43 PM To: [EMAIL PROTECTED] Subject: Re: low bandwitdth and big files From: Matthew Glanville You might want to turn on TSM client side compression... In my experience notes databases can get at least 50% compressed. Your backups will most likely go down to 2 hours, or even more. TSM: update node node_name compress=yes Give it a try. For low bandwith lines I always prefer to let TSM compress the data first Of course, we no longer back up notes as normal files but use the TDP for Domino agent
Re: Antwort: Re: low bandwitdth and big files
Nick, I have client compression turned on also due to slow network (have no choice). But no one has been able to answer the following questions definitively: 1. Am I potentially doubling the size of certain files in the stg pool by running multiple compression algorithms.? 2. By turning off DEVCLASS compression, is that effectively disabling hardware compression performed by my tape device (IBM 3590 TAPE Device / Cartridge) 3. If client compression and hardware compression are turned on, and hardware compression isn't really buying me anything... won't the attempt at hardware compression prevent streaming? I think it will. I'm looking for YES/NO answers with a valid explanation. Anyone? Regards, Joe -Original Message- From: Nicholas Cassimatis [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 31, 2002 1:32 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: Re: low bandwitdth and big files A long while back, I had 36 boxes of the following config: Pentium 100's, 128MB RAM, 16Mbit Token Ring, running OS/2 2.11 with Lan Server 4, Notes 4.1, backing up mail files as flat files. Turned client side compression on, backup window went from 4 hours to 1.25 hours. I cut the data sent over the wire down by 66%, and got a corresponding reduction in the backup time. My machines were effectively offline for the backup window, due to the network being saturated, so the fact they were also CPU bound really didn't matter. It all depends on the config. The worst you could do is to test a little, see what happens. Oh, my library was a 3494 with 2xB11 drives in it. I kept utilizing the same number of tapes, but the capacity at full went from around 28GB to 11GB, as would be expected. Nick Cassimatis Technical Team Lead e-Business Backup/Recovery Services 919-363-8894 T/L 223-8965 [EMAIL PROTECTED] Today is the tomorrow of yesterday.
Re: Antwort: Re: low bandwitdth and big files
1. Yes. Trying to compress an already compressed file can cause the file to grow larger. 2. No. We run a 3494 Magstar - hardware compression is ALWAYS on, you can't override it. It is controlled within the hardware and our ZOS system. 3. Prevent - No. Degrade or hinder - Yes, it can. -Original Message- From: Wholey, Joseph (TGA\MLOL) [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 31, 2002 1:05 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: Re: low bandwitdth and big files Nick, I have client compression turned on also due to slow network (have no choice). But no one has been able to answer the following questions definitively: 1. Am I potentially doubling the size of certain files in the stg pool by running multiple compression algorithms.? 2. By turning off DEVCLASS compression, is that effectively disabling hardware compression performed by my tape device (IBM 3590 TAPE Device / Cartridge) 3. If client compression and hardware compression are turned on, and hardware compression isn't really buying me anything... won't the attempt at hardware compression prevent streaming? I think it will. I'm looking for YES/NO answers with a valid explanation. Anyone? Regards, Joe -Original Message- From: Nicholas Cassimatis [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 31, 2002 1:32 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: Re: low bandwitdth and big files A long while back, I had 36 boxes of the following config: Pentium 100's, 128MB RAM, 16Mbit Token Ring, running OS/2 2.11 with Lan Server 4, Notes 4.1, backing up mail files as flat files. Turned client side compression on, backup window went from 4 hours to 1.25 hours. I cut the data sent over the wire down by 66%, and got a corresponding reduction in the backup time. My machines were effectively offline for the backup window, due to the network being saturated, so the fact they were also CPU bound really didn't matter. It all depends on the config. The worst you could do is to test a little, see what happens. Oh, my library was a 3494 with 2xB11 drives in it. I kept utilizing the same number of tapes, but the capacity at full went from around 28GB to 11GB, as would be expected. Nick Cassimatis Technical Team Lead e-Business Backup/Recovery Services 919-363-8894 T/L 223-8965 [EMAIL PROTECTED] Today is the tomorrow of yesterday.
Re: Antwort: Re: low bandwitdth and big files
HOO---RAAY... finally. Thank you. -Original Message- From: Slag, Jerry B. [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 31, 2002 2:37 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: Re: low bandwitdth and big files 1. Yes. Trying to compress an already compressed file can cause the file to grow larger. 2. No. We run a 3494 Magstar - hardware compression is ALWAYS on, you can't override it. It is controlled within the hardware and our ZOS system. 3. Prevent - No. Degrade or hinder - Yes, it can. -Original Message- From: Wholey, Joseph (TGA\MLOL) [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 31, 2002 1:05 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: Re: low bandwitdth and big files Nick, I have client compression turned on also due to slow network (have no choice). But no one has been able to answer the following questions definitively: 1. Am I potentially doubling the size of certain files in the stg pool by running multiple compression algorithms.? 2. By turning off DEVCLASS compression, is that effectively disabling hardware compression performed by my tape device (IBM 3590 TAPE Device / Cartridge) 3. If client compression and hardware compression are turned on, and hardware compression isn't really buying me anything... won't the attempt at hardware compression prevent streaming? I think it will. I'm looking for YES/NO answers with a valid explanation. Anyone? Regards, Joe -Original Message- From: Nicholas Cassimatis [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 31, 2002 1:32 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: Re: low bandwitdth and big files A long while back, I had 36 boxes of the following config: Pentium 100's, 128MB RAM, 16Mbit Token Ring, running OS/2 2.11 with Lan Server 4, Notes 4.1, backing up mail files as flat files. Turned client side compression on, backup window went from 4 hours to 1.25 hours. I cut the data sent over the wire down by 66%, and got a corresponding reduction in the backup time. My machines were effectively offline for the backup window, due to the network being saturated, so the fact they were also CPU bound really didn't matter. It all depends on the config. The worst you could do is to test a little, see what happens. Oh, my library was a 3494 with 2xB11 drives in it. I kept utilizing the same number of tapes, but the capacity at full went from around 28GB to 11GB, as would be expected. Nick Cassimatis Technical Team Lead e-Business Backup/Recovery Services 919-363-8894 T/L 223-8965 [EMAIL PROTECTED] Today is the tomorrow of yesterday.
Re: Antwort: Re: low bandwitdth and big files
1. Am I potentially doubling the size of certain files in the stg pool by running multiple compression algorithms.? No - the drive logic won't compress already compressed data, so there's no real risk of this. 2. By turning off DEVCLASS compression, is that effectively disabling hardware compression performed by my tape device (IBM 3590 TAPE Device / Cartridge) Yes. 3. If client compression and hardware compression are turned on, and hardware compression isn't really buying me anything... won't the attempt at hardware compression prevent streaming? I think it will. Nope, not to the best of my knowledge. In fact, a pre-compressed data stream is less likely to starve the drive. I'm looking for YES/NO answers with a valid explanation. Anyone? Nick Cassimatis [EMAIL PROTECTED] Today is the tomorrow of yesterday.
Re: Antwort: Re: low bandwitdth and big files
1. Hardware microcode is smart - software can be stupid. Using compression yes and /or compressalways yes can and will cause some files to increase in size. 2. We have compression turned on within the 3494, within our ZOS operating system and within System Managed Storage settings. If any of these are on it does not matter what TSM tries to do - these settings override. Obviously if you do not have anything controlling compression at a higher level TSM may. I believe the hardware default for 3590 drives is to have compression on. -Original Message- From: Nicholas Cassimatis [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 31, 2002 1:42 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: Re: low bandwitdth and big files 1. Am I potentially doubling the size of certain files in the stg pool by running multiple compression algorithms.? No - the drive logic won't compress already compressed data, so there's no real risk of this. 2. By turning off DEVCLASS compression, is that effectively disabling hardware compression performed by my tape device (IBM 3590 TAPE Device / Cartridge) Yes. 3. If client compression and hardware compression are turned on, and hardware compression isn't really buying me anything... won't the attempt at hardware compression prevent streaming? I think it will. Nope, not to the best of my knowledge. In fact, a pre-compressed data stream is less likely to starve the drive. I'm looking for YES/NO answers with a valid explanation. Anyone? Nick Cassimatis [EMAIL PROTECTED] Today is the tomorrow of yesterday.
