Re: How to figure collocation overhead
Don't forget collocation by group. All of our nodes belong to one group or another and we can then be pretty sure that collocation will do what we need. Mostly, our tape pools are collocated by group. Some nodes are in a group all by themselves and some (e.g. workstations) have many nodes in their group. We only use collocation by filespace if it is really needed for the largest nodes. Bob -Original Message- From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf Of Sam Rawlins Sent: Thursday, January 15, 2009 2:20 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] How to figure collocation overhead Hi Bill, I like Wanda's point. MAXSCRatch is the best way to control tape usage. You can probably write a fun select to see how many nodes have data on the average tape. In Wanda's example, there are about 2 nodes per tape. If you see that that number gets too high, you can increase MAXSCRatch. On Thu, Jan 15, 2009 at 11:53 AM, Wanda Prather wanda.prat...@jasi.comwrote: Collocation consumes as much of the library as you tell it to. When you set MAXSCRATCH on a storage pool, TSM will confine the pool to that number of tapes. If you have less tapes available than you have clients, you get a do the best you can collocation. There are plenty of sites who have more clients than slots in their library. If you have 100 clients and set MAXSCRATCH for the pool to 50 tapes, TSM will put each client on a different tape until it hits 50, then it will double up. You still get a decent amount of collocation; it's a lot faster for a restore to skip over another clients data on the same tape, than it is to rewind, dismount, mount, and reposition to a different tape. What you have to remember if you are collocating, is that you have to check your pools periodically and reevaluate maxscratch for the pool. If your total amount of data is growing, you will need to up the maxscratch periodically so that you maintain enough available free tapes in the pool to provide a reasonable level of collocation. W On Thu, Jan 15, 2009 at 1:34 PM, Evans, Bill bev...@fhcrc.org wrote: Is there a guide or rule of thumb for determining what collocation will do to library space? It is being considered for improving restore speeds. I'm backing up 80TB in 30 volumes(filespaces) on a solaris server. Currently there is no collocation set, all data goes to a long-term management class, single onsite and off-site tape pools, data churn averages 1TB/night. Using a 650 slot L700 library, LTO3, TSM 5.5, AIX 5.3. The library is 80% full, so, I'm concerned that starting collocation for the volumes will consume the rest of the library. Each filespace collocation may end up with one tape not full (or 30 in this case). Thanks, Bill Evans Research Computing Support FRED HUTCHINSON CANCER RESEARCH CENTER -- Sam Rawlins This electronic transmission and any documents accompanying this electronic transmission contain confidential information belonging to the sender. This information may be legally privileged. The information is intended only for the use of the individual or entity named above. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or the taking of any action in reliance on or regarding the contents of this electronically transmitted information is strictly prohibited.
Re: How to figure collocation overhead
Thanks for all the advice. Now to go try some things. I'll update the list when I have some data. Thank you, Bill Evans Research Computing Support FRED HUTCHINSON CANCER RESEARCH CENTER -Original Message- From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf Of Bob Levad Sent: Friday, January 16, 2009 7:52 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] How to figure collocation overhead Don't forget collocation by group. All of our nodes belong to one group or another and we can then be pretty sure that collocation will do what we need. Mostly, our tape pools are collocated by group. Some nodes are in a group all by themselves and some (e.g. workstations) have many nodes in their group. We only use collocation by filespace if it is really needed for the largest nodes. Bob -Original Message- From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf Of Sam Rawlins Sent: Thursday, January 15, 2009 2:20 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] How to figure collocation overhead Hi Bill, I like Wanda's point. MAXSCRatch is the best way to control tape usage. You can probably write a fun select to see how many nodes have data on the average tape. In Wanda's example, there are about 2 nodes per tape. If you see that that number gets too high, you can increase MAXSCRatch. On Thu, Jan 15, 2009 at 11:53 AM, Wanda Prather wanda.prat...@jasi.comwrote: Collocation consumes as much of the library as you tell it to. When you set MAXSCRATCH on a storage pool, TSM will confine the pool to that number of tapes. If you have less tapes available than you have clients, you get a do the best you can collocation. There are plenty of sites who have more clients than slots in their library. If you have 100 clients and set MAXSCRATCH for the pool to 50 tapes, TSM will put each client on a different tape until it hits 50, then it will double up. You still get a decent amount of collocation; it's a lot faster for a restore to skip over another clients data on the same tape, than it is to rewind, dismount, mount, and reposition to a different tape. What you have to remember if you are collocating, is that you have to check your pools periodically and reevaluate maxscratch for the pool. If your total amount of data is growing, you will need to up the maxscratch periodically so that you maintain enough available free tapes in the pool to provide a reasonable level of collocation. W On Thu, Jan 15, 2009 at 1:34 PM, Evans, Bill bev...@fhcrc.org wrote: Is there a guide or rule of thumb for determining what collocation will do to library space? It is being considered for improving restore speeds. I'm backing up 80TB in 30 volumes(filespaces) on a solaris server. Currently there is no collocation set, all data goes to a long-term management class, single onsite and off-site tape pools, data churn averages 1TB/night. Using a 650 slot L700 library, LTO3, TSM 5.5, AIX 5.3. The library is 80% full, so, I'm concerned that starting collocation for the volumes will consume the rest of the library. Each filespace collocation may end up with one tape not full (or 30 in this case). Thanks, Bill Evans Research Computing Support FRED HUTCHINSON CANCER RESEARCH CENTER -- Sam Rawlins This electronic transmission and any documents accompanying this electronic transmission contain confidential information belonging to the sender. This information may be legally privileged. The information is intended only for the use of the individual or entity named above. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or the taking of any action in reliance on or regarding the contents of this electronically transmitted information is strictly prohibited.
How to figure collocation overhead
Is there a guide or rule of thumb for determining what collocation will do to library space? It is being considered for improving restore speeds. I'm backing up 80TB in 30 volumes(filespaces) on a solaris server. Currently there is no collocation set, all data goes to a long-term management class, single onsite and off-site tape pools, data churn averages 1TB/night. Using a 650 slot L700 library, LTO3, TSM 5.5, AIX 5.3. The library is 80% full, so, I'm concerned that starting collocation for the volumes will consume the rest of the library. Each filespace collocation may end up with one tape not full (or 30 in this case). Thanks, Bill Evans Research Computing Support FRED HUTCHINSON CANCER RESEARCH CENTER
Re: How to figure collocation overhead
You can easily determine how many lines of tapes you will initially break out into, based upon which of the multiple types of collocation you ultimately decide upon and how many nodes or filespaces are active, sending data to TSM storage. I would double that number to account for tape leak, based upon historic experience. And if there are Windows clients, think more about DIRMc in that context. I think most of us implement collocation judiciously, based partly upon library cell impact, but probably more so on realities of tape drive contention. A big arrival disk pool facilitates more realistic collocation. Richard Sims
Re: How to figure collocation overhead
Collocation consumes as much of the library as you tell it to. When you set MAXSCRATCH on a storage pool, TSM will confine the pool to that number of tapes. If you have less tapes available than you have clients, you get a do the best you can collocation. There are plenty of sites who have more clients than slots in their library. If you have 100 clients and set MAXSCRATCH for the pool to 50 tapes, TSM will put each client on a different tape until it hits 50, then it will double up. You still get a decent amount of collocation; it's a lot faster for a restore to skip over another clients data on the same tape, than it is to rewind, dismount, mount, and reposition to a different tape. What you have to remember if you are collocating, is that you have to check your pools periodically and reevaluate maxscratch for the pool. If your total amount of data is growing, you will need to up the maxscratch periodically so that you maintain enough available free tapes in the pool to provide a reasonable level of collocation. W On Thu, Jan 15, 2009 at 1:34 PM, Evans, Bill bev...@fhcrc.org wrote: Is there a guide or rule of thumb for determining what collocation will do to library space? It is being considered for improving restore speeds. I'm backing up 80TB in 30 volumes(filespaces) on a solaris server. Currently there is no collocation set, all data goes to a long-term management class, single onsite and off-site tape pools, data churn averages 1TB/night. Using a 650 slot L700 library, LTO3, TSM 5.5, AIX 5.3. The library is 80% full, so, I'm concerned that starting collocation for the volumes will consume the rest of the library. Each filespace collocation may end up with one tape not full (or 30 in this case). Thanks, Bill Evans Research Computing Support FRED HUTCHINSON CANCER RESEARCH CENTER
Re: How to figure collocation overhead
Hi Bill, I like Wanda's point. MAXSCRatch is the best way to control tape usage. You can probably write a fun select to see how many nodes have data on the average tape. In Wanda's example, there are about 2 nodes per tape. If you see that that number gets too high, you can increase MAXSCRatch. On Thu, Jan 15, 2009 at 11:53 AM, Wanda Prather wanda.prat...@jasi.comwrote: Collocation consumes as much of the library as you tell it to. When you set MAXSCRATCH on a storage pool, TSM will confine the pool to that number of tapes. If you have less tapes available than you have clients, you get a do the best you can collocation. There are plenty of sites who have more clients than slots in their library. If you have 100 clients and set MAXSCRATCH for the pool to 50 tapes, TSM will put each client on a different tape until it hits 50, then it will double up. You still get a decent amount of collocation; it's a lot faster for a restore to skip over another clients data on the same tape, than it is to rewind, dismount, mount, and reposition to a different tape. What you have to remember if you are collocating, is that you have to check your pools periodically and reevaluate maxscratch for the pool. If your total amount of data is growing, you will need to up the maxscratch periodically so that you maintain enough available free tapes in the pool to provide a reasonable level of collocation. W On Thu, Jan 15, 2009 at 1:34 PM, Evans, Bill bev...@fhcrc.org wrote: Is there a guide or rule of thumb for determining what collocation will do to library space? It is being considered for improving restore speeds. I'm backing up 80TB in 30 volumes(filespaces) on a solaris server. Currently there is no collocation set, all data goes to a long-term management class, single onsite and off-site tape pools, data churn averages 1TB/night. Using a 650 slot L700 library, LTO3, TSM 5.5, AIX 5.3. The library is 80% full, so, I'm concerned that starting collocation for the volumes will consume the rest of the library. Each filespace collocation may end up with one tape not full (or 30 in this case). Thanks, Bill Evans Research Computing Support FRED HUTCHINSON CANCER RESEARCH CENTER -- Sam Rawlins
