Re: Slow tape to tape performance
FYI. Quoted from the Admin Ref:
"MOVESizethresh megabytes
Parameters
megabytes
Specifies the number of megabytes as an integer from 1 to 2048. "
Zlatko Krastev
IT Consultant
"Wilcox, Andy" <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
27.12.2002 13:46
Please respond to "ADSM: Dist Stor Manager"
To: [EMAIL PROTECTED]
cc:
Subject:Re: Slow tape to tape performance
Firstly I must ask people to accept my apologisies if this message doesn't
look right This is first time I have tried to post a message hear :-)
Just reading through a scenario posted by Michael Wheelock about slow tape
to tape perforance... There are lots of considerations here to take note
of...
We run with an P660-6M1 (near enough a similar server) with a 3584
attached
to four HVD SCSI LTO Drives and four FC LTO Drives (over a SAN). Each SCSI
drive is attached directly to its own SCSI card in the server to max the
performance. With the FC drives we can see all four over each of the four
FC
adapters in the server. You need to make sure you use the as few drives on
one FC bus as possible...
+ fcs1 51-08 FC Adapter
* fscsi151-08-01 FC SCSI I/O Controller Protocol
+ rmt8 51-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ rmt9 51-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ rmt10 51-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ rmt11 51-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ fcs2 61-08 FC Adapter
* fscsi261-08-01 FC SCSI I/O Controller Protocol
+ rmt12 61-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ rmt13 61-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ rmt14 61-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ rmt15 61-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ fcs3 71-08 FC Adapter
* fscsi371-08-01 FC SCSI I/O Controller Protocol
+ rmt16 71-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ rmt17 71-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ rmt18 71-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ rmt19 71-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ fcs0 21-08 FC Adapter
* fscsi021-08-01 FC SCSI I/O Controller Protocol
+ rmt4 21-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ rmt5 21-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ rmt6 21-08-01 IBM 3580 Ultrium Tape Drive (FCP)
+ rmt7 21-08-01 IBM 3580 Ultrium Tape Drive (FCP)
In the example above using one drive for my explaination rmt8, rmt12,
rmt16
and rmt4 are the same physical drive seen across four FC paths. so in this
case I would pick rmt8. Second drive can be seen as rmt9, rmt13, rmt17 and
rmt5 so I use rmt13 and so on.
If you run like this then you are maxing the throughput available to each
drive Please note the IBM recommendation is no more than 2 drives per
FC
adapter.
With the above configuration we achieve tape-to-tape of around 40-80GB an
hour (depending on data and client/server compression).
A couple of other tweaks we have set in dsmserv.opt
TXNGroupmax 256
MOVEBatchsize 1000
MOVESizethresh 5000
Finally on the devclass we have set the format to ultriumc.
Let me know how you get on
Cheers
Andy Wilcox
Confidentiality: This e-mail and any files transmitted with it are
confidential and intended solely for the use of the individual or entity
to whom they are addressed. If you have received this e-mail in error,
use of this information (including disclosure, copying or distribution)
may be unlawful. Please notify [EMAIL PROTECTED] and
delete the message immediately.
Security: Internet e-mail is not a 100% secure communications medium.
Viruses: This e-mail (and any attachments) has been checked (using Sophos
Sweep 3.63 + patches) and found to be clean from any virus infection
before leaving.
Therefore neither Aquila Networks Services Ltd nor Midlands Electricity
plc or any of their group undertakings (as defined by the Companies Act
1989) (together referred to as the "Companies") accept legal
responsibility for this message or liability for the consequences of any
computer viruses which may have been transmitted by this e-mail.
Monitoring: All electronic communications with the Companies may be
monitored in accordance with the UK Regulation of Investigatory Powers
Act, Lawful Business Practice Regulations, 2000. If you do not consent to
such monitoring, you should contact the sender of this e-mail.
Aquila Networks Services Limited,
R
Re: Slow tape to tape performance
Firstly I must ask people to accept my apologisies if this message doesn't look right This is first time I have tried to post a message hear :-) Just reading through a scenario posted by Michael Wheelock about slow tape to tape perforance... There are lots of considerations here to take note of... We run with an P660-6M1 (near enough a similar server) with a 3584 attached to four HVD SCSI LTO Drives and four FC LTO Drives (over a SAN). Each SCSI drive is attached directly to its own SCSI card in the server to max the performance. With the FC drives we can see all four over each of the four FC adapters in the server. You need to make sure you use the as few drives on one FC bus as possible... + fcs1 51-08 FC Adapter * fscsi151-08-01 FC SCSI I/O Controller Protocol + rmt8 51-08-01 IBM 3580 Ultrium Tape Drive (FCP) + rmt9 51-08-01 IBM 3580 Ultrium Tape Drive (FCP) + rmt10 51-08-01 IBM 3580 Ultrium Tape Drive (FCP) + rmt11 51-08-01 IBM 3580 Ultrium Tape Drive (FCP) + fcs2 61-08 FC Adapter * fscsi261-08-01 FC SCSI I/O Controller Protocol + rmt12 61-08-01 IBM 3580 Ultrium Tape Drive (FCP) + rmt13 61-08-01 IBM 3580 Ultrium Tape Drive (FCP) + rmt14 61-08-01 IBM 3580 Ultrium Tape Drive (FCP) + rmt15 61-08-01 IBM 3580 Ultrium Tape Drive (FCP) + fcs3 71-08 FC Adapter * fscsi371-08-01 FC SCSI I/O Controller Protocol + rmt16 71-08-01 IBM 3580 Ultrium Tape Drive (FCP) + rmt17 71-08-01 IBM 3580 Ultrium Tape Drive (FCP) + rmt18 71-08-01 IBM 3580 Ultrium Tape Drive (FCP) + rmt19 71-08-01 IBM 3580 Ultrium Tape Drive (FCP) + fcs0 21-08 FC Adapter * fscsi021-08-01 FC SCSI I/O Controller Protocol + rmt4 21-08-01 IBM 3580 Ultrium Tape Drive (FCP) + rmt5 21-08-01 IBM 3580 Ultrium Tape Drive (FCP) + rmt6 21-08-01 IBM 3580 Ultrium Tape Drive (FCP) + rmt7 21-08-01 IBM 3580 Ultrium Tape Drive (FCP) In the example above using one drive for my explaination rmt8, rmt12, rmt16 and rmt4 are the same physical drive seen across four FC paths. so in this case I would pick rmt8. Second drive can be seen as rmt9, rmt13, rmt17 and rmt5 so I use rmt13 and so on. If you run like this then you are maxing the throughput available to each drive Please note the IBM recommendation is no more than 2 drives per FC adapter. With the above configuration we achieve tape-to-tape of around 40-80GB an hour (depending on data and client/server compression). A couple of other tweaks we have set in dsmserv.opt TXNGroupmax 256 MOVEBatchsize 1000 MOVESizethresh 5000 Finally on the devclass we have set the format to ultriumc. Let me know how you get on Cheers Andy Wilcox Confidentiality: This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, use of this information (including disclosure, copying or distribution) may be unlawful. Please notify [EMAIL PROTECTED] and delete the message immediately. Security: Internet e-mail is not a 100% secure communications medium. Viruses: This e-mail (and any attachments) has been checked (using Sophos Sweep 3.63 + patches) and found to be clean from any virus infection before leaving. Therefore neither Aquila Networks Services Ltd nor Midlands Electricity plc or any of their group undertakings (as defined by the Companies Act 1989) (together referred to as the "Companies") accept legal responsibility for this message or liability for the consequences of any computer viruses which may have been transmitted by this e-mail. Monitoring: All electronic communications with the Companies may be monitored in accordance with the UK Regulation of Investigatory Powers Act, Lawful Business Practice Regulations, 2000. If you do not consent to such monitoring, you should contact the sender of this e-mail. Aquila Networks Services Limited, Registered office: Whittington Hall, Whittington, Worcester, WR5 2RB Registered in England and Wales number 3600545 This e-mail may be sent on behalf of any of the Companies.
Re: Slow tape to tape performance...
>I am getting agonizingly slow (~1GB/hr) performance on tape to tape copies >(such as during reclamations). What experiments have you performed to try to narrow down the problem? What results do you see in an experiment where data is copied between the same two tape drives outside of TSM? Given the cost of FC adapters, is it the case that you have one, and you are seeing "tidal action" as drive-to-drive copies are performed? As Kelly suggested, the driver level for the FC adapter may greatly affect performance. I presume that you are seeing this performance problem with all tapes? Marginal tape samples can result in a lot of retries, and degraded performance. Don't overlook microcode level on the drives: some mc levels are bow-wows. Richard Sims, BU
Re: Slow tape to tape performance...
Performance is good when writing from disk to tape? How good? When I hear of this I suspect driver issues. I know on Win2K driver selection is paramount when having a performance problem with tape. Kelly J. Lipp STORServer, Inc. 485-B Elkton Drive Colorado Springs, CO 80907 [EMAIL PROTECTED] or [EMAIL PROTECTED] www.storsol.com or www.storserver.com (719)531-5926 Fax: (240)539-7175 -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Remeta, Mark Sent: Tuesday, December 24, 2002 10:08 AM To: [EMAIL PROTECTED] Subject: Re: Slow tape to tape performance... do you have tape drive compression and client compression turned on? I've seen this before with AIT drives trying to compress already compressed data... -Original Message- From: Wheelock, Michael D [mailto:[EMAIL PROTECTED]] Sent: Tuesday, December 24, 2002 11:33 AM To: [EMAIL PROTECTED] Subject: Slow tape to tape performance... Hi, I am getting agonizingly slow (~1GB/hr) performance on tape to tape copies (such as during reclamations). Is this normal? This is a new install, fairly fresh with some basic policies and copy groups. The server is an AIX 5.1 ML02 7026-M80 with 8GB RAM. The tape library is an IBM 3584 with 8 FC attached LTO drives. TSM is at 5.1.5.4. Any ideas would be appreciated. Michael Wheelock Integris Health of Oklahoma Confidentiality Note: The information transmitted is intended only for the person or entity to whom or which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please delete this material immediately.
Re: Slow tape to tape performance...
do you have tape drive compression and client compression turned on? I've seen this before with AIT drives trying to compress already compressed data... -Original Message- From: Wheelock, Michael D [mailto:[EMAIL PROTECTED]] Sent: Tuesday, December 24, 2002 11:33 AM To: [EMAIL PROTECTED] Subject: Slow tape to tape performance... Hi, I am getting agonizingly slow (~1GB/hr) performance on tape to tape copies (such as during reclamations). Is this normal? This is a new install, fairly fresh with some basic policies and copy groups. The server is an AIX 5.1 ML02 7026-M80 with 8GB RAM. The tape library is an IBM 3584 with 8 FC attached LTO drives. TSM is at 5.1.5.4. Any ideas would be appreciated. Michael Wheelock Integris Health of Oklahoma Confidentiality Note: The information transmitted is intended only for the person or entity to whom or which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please delete this material immediately.
Slow tape to tape performance...
Hi, I am getting agonizingly slow (~1GB/hr) performance on tape to tape copies (such as during reclamations). Is this normal? This is a new install, fairly fresh with some basic policies and copy groups. The server is an AIX 5.1 ML02 7026-M80 with 8GB RAM. The tape library is an IBM 3584 with 8 FC attached LTO drives. TSM is at 5.1.5.4. Any ideas would be appreciated. Michael Wheelock Integris Health of Oklahoma
Re: tape-to-tape performance
What sort of performance are you getting? In Gbyte/hour. I would expect somewhere in the 40-60 GB/hour range. Kelly J. Lipp STORServer, Inc. 485-B Elkton Drive Colorado Springs, CO 80907 [EMAIL PROTECTED] or [EMAIL PROTECTED] www.storsol.com or www.storserver.com (719)531-5926 Fax: (240)539-7175 -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Joshua Bassi Sent: Wednesday, November 20, 2002 1:26 PM To: [EMAIL PROTECTED] Subject: Re: tape-to-tape performance Take a look at the movebatchsize, movesizethresh and bufpoolsize. These 3 options have a direct corrolation for tape to tape copy processes. -- Joshua S. Bassi IBM Certified - AIX 4/5L, SAN, Shark Tivoli Certified Consultant - ADSM/TSM eServer Systems Expert -pSeries HACMP AIX, HACMP, Storage, TSM Consultant Cell (831) 595-3962 [EMAIL PROTECTED] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]] On Behalf Of Conko, Steven Sent: Wednesday, November 20, 2002 11:26 AM To: [EMAIL PROTECTED] Subject: tape-to-tape performance we run a 3494 tape library with 3590 tape drives. after our backup are finished, we run a backup stgpool to a copy pool on about 1.6TB of data. as you can imagine, even with 8 drives this takes some time. are there any server, device or other parameters we can tune to improve the tape-to-tape performance? steve
Re: tape-to-tape performance
As much of a performance hit as you want ! With about every TSM task (straight backups via "resourceutilization" or with tdp/r3 with the number of concurrent sessions, etc...) you can tune how busy TSM will keep your system. We have our production boxes at one site and our disaster recovery center miles away, our TSM server(s) are located at the DR site, we have to compress all client data to send it across the OC12 between the sites. For example, we can crank up TSM activities to suck up 90% of 20 processors (out of 32) on a Sun E10K by using client compression... So an OC12 is 622 Mb/sec or roughly 273 GB/hr, on a nightly basis we see it running somewhere between 175-200 GB/hr so we still have about 1/3 capacity BUT we see 3.8/1 compression of most data base data so we are actually moving somewhere around 760 GB/hr of client filespace and I'd hate to be pay'n for an OC24 (or more) between the sites (if we could even get it) ! One view is that if you beef up the processors to assist in backup, the end results to the user community is increased performance during their production use ;-) It's a (god I hat to say this) "WIN ! WIN ! situation... Just remind management that today they are more than ever likely to have to depend on their DR ! Dwight -Original Message- From: Conko, Steven [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 20, 2002 2:35 PM To: [EMAIL PROTECTED] Subject: Re: tape-to-tape performance thanks for that insight. i forgot to mention we are running fibre channel and not scsi drives. yes, the data is must have... it is all production data and is being copied for DRM. how much of a performance hit will it take to turn compression on at the client level? -Original Message- From: Cook, Dwight E [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 20, 2002 3:22 PM To: [EMAIL PROTECTED] Subject: Re: tape-to-tape performance Is 1.6 TB the amount of "must have/critical" information ? and is that already compressed ? Knowing each tape mount runs about 90 seconds, any way to reduce tape mounts will speed up copies. If you have collocation on, turning it off ~could~ help... and collocation can/is set on both primary pools and copy pools. Now if the data isn't compressed by the client the data is uncompressed at the drive, moved through the processor as ~full size~ data, then recompressed at the destination drive. If your clients have the horsepower, turn on client compression, that way not so much data is moved across the processor's buss. Don't daisy chain any of your 3590's if they are older scsi. You might be able to reduce your data down if you force the application owners to review their required CRITICAL/MUST HAVE data and send it to isolated pools for copies to take offsite. just some thoughts... Dwight -Original Message- From: Conko, Steven [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 20, 2002 1:26 PM To: [EMAIL PROTECTED] Subject: tape-to-tape performance we run a 3494 tape library with 3590 tape drives. after our backup are finished, we run a backup stgpool to a copy pool on about 1.6TB of data. as you can imagine, even with 8 drives this takes some time. are there any server, device or other parameters we can tune to improve the tape-to-tape performance? steve
Re: tape-to-tape performance
thanks for that insight. i forgot to mention we are running fibre channel and not scsi drives. yes, the data is must have... it is all production data and is being copied for DRM. how much of a performance hit will it take to turn compression on at the client level? -Original Message- From: Cook, Dwight E [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 20, 2002 3:22 PM To: [EMAIL PROTECTED] Subject: Re: tape-to-tape performance Is 1.6 TB the amount of "must have/critical" information ? and is that already compressed ? Knowing each tape mount runs about 90 seconds, any way to reduce tape mounts will speed up copies. If you have collocation on, turning it off ~could~ help... and collocation can/is set on both primary pools and copy pools. Now if the data isn't compressed by the client the data is uncompressed at the drive, moved through the processor as ~full size~ data, then recompressed at the destination drive. If your clients have the horsepower, turn on client compression, that way not so much data is moved across the processor's buss. Don't daisy chain any of your 3590's if they are older scsi. You might be able to reduce your data down if you force the application owners to review their required CRITICAL/MUST HAVE data and send it to isolated pools for copies to take offsite. just some thoughts... Dwight -Original Message- From: Conko, Steven [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 20, 2002 1:26 PM To: [EMAIL PROTECTED] Subject: tape-to-tape performance we run a 3494 tape library with 3590 tape drives. after our backup are finished, we run a backup stgpool to a copy pool on about 1.6TB of data. as you can imagine, even with 8 drives this takes some time. are there any server, device or other parameters we can tune to improve the tape-to-tape performance? steve
Re: tape-to-tape performance
Is 1.6 TB the amount of "must have/critical" information ? and is that already compressed ? Knowing each tape mount runs about 90 seconds, any way to reduce tape mounts will speed up copies. If you have collocation on, turning it off ~could~ help... and collocation can/is set on both primary pools and copy pools. Now if the data isn't compressed by the client the data is uncompressed at the drive, moved through the processor as ~full size~ data, then recompressed at the destination drive. If your clients have the horsepower, turn on client compression, that way not so much data is moved across the processor's buss. Don't daisy chain any of your 3590's if they are older scsi. You might be able to reduce your data down if you force the application owners to review their required CRITICAL/MUST HAVE data and send it to isolated pools for copies to take offsite. just some thoughts... Dwight -Original Message- From: Conko, Steven [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 20, 2002 1:26 PM To: [EMAIL PROTECTED] Subject: tape-to-tape performance we run a 3494 tape library with 3590 tape drives. after our backup are finished, we run a backup stgpool to a copy pool on about 1.6TB of data. as you can imagine, even with 8 drives this takes some time. are there any server, device or other parameters we can tune to improve the tape-to-tape performance? steve
Re: tape-to-tape performance
Take a look at the movebatchsize, movesizethresh and bufpoolsize. These 3 options have a direct corrolation for tape to tape copy processes. -- Joshua S. Bassi IBM Certified - AIX 4/5L, SAN, Shark Tivoli Certified Consultant - ADSM/TSM eServer Systems Expert -pSeries HACMP AIX, HACMP, Storage, TSM Consultant Cell (831) 595-3962 [EMAIL PROTECTED] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]] On Behalf Of Conko, Steven Sent: Wednesday, November 20, 2002 11:26 AM To: [EMAIL PROTECTED] Subject: tape-to-tape performance we run a 3494 tape library with 3590 tape drives. after our backup are finished, we run a backup stgpool to a copy pool on about 1.6TB of data. as you can imagine, even with 8 drives this takes some time. are there any server, device or other parameters we can tune to improve the tape-to-tape performance? steve
Re: tape-to-tape performance
The storage pool backup that you are doing could be directly affected by the database performance. Is this 1.6TB a lot of small files? On the IO configuration side, how are your tape drives connected, SCSI or FC? I suspect SCSI. If you have more than 2 drives, on a SCSI adapter that is the problem. If they are E-model they really should be 2 per adapter to get descent performance. On a FC scenario you can go up to 4 drives per FC adapter and have very good performance. I have gone has high as 8 per FC adapter, but performance does suffer then. If you are using FC, make sure your FC tape and disk are not on the same adapters on a TSM server. Other applications it is fine, but not this one. If your TSM database is on slow disk that will slow down the copies. It has to create a lot of updates to the database to copy the storage pools. Paul D. Seay, Jr. Technical Specialist Naptheon Inc. 757-688-8180 -Original Message- From: Conko, Steven [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 20, 2002 2:26 PM To: [EMAIL PROTECTED] Subject: tape-to-tape performance we run a 3494 tape library with 3590 tape drives. after our backup are finished, we run a backup stgpool to a copy pool on about 1.6TB of data. as you can imagine, even with 8 drives this takes some time. are there any server, device or other parameters we can tune to improve the tape-to-tape performance? steve
tape-to-tape performance
we run a 3494 tape library with 3590 tape drives. after our backup are finished, we run a backup stgpool to a copy pool on about 1.6TB of data. as you can imagine, even with 8 drives this takes some time. are there any server, device or other parameters we can tune to improve the tape-to-tape performance? steve
