Re: Collocation anomaly report
On 04/16/2013 08:15 PM, Nick Laflamme wrote: If you absolutely need for nodes to be isolated on their own media, why aren't they in their own individual domains which point to their own storage pools, all of which might share a library? That'd work; You could also make it an absolute requirement of your infrastructure to maintain scratch availability, and do new acquisitions when scratch levels fall below foo. Frankly, I like that TSM will override collocation preferences when its at MAXSCR for volumes in a pool. Me too. I have a script, which I think I've bandied about here in the past, which evaluates collocation for me, but I think it's poorly aligned with your needs, Grant: I select candidates for inclusion into collocation groups, and then once that's done I MOVE DATA to vacate tapes which are low-utilization and have a candidate collocated home. - Allen S. Rout
Re: Collocation anomaly report
On 04/16/2013 09:26 PM, Grant Street wrote: I understand that getting a good backup is the most important, but when you offer a feature that is best efforts with no way to verify, why have the feature? Your conceptualization of the feature is inverted from the conceptualization under which it was implemented. I view collocation as a tool to Keep data from a given node in physical proximity, to reduce mount latency at restore time. Any of you who have attempted to service restores from a large pool of uncollocated data (raises hand) would see this as the primary purpose of the tool, too. (mutter mutter read docs). There: 'help def stg': Collocation reduces the number of sequential access media mounts for restore, retrieve, and recall operations. So collocation accomplishes that feature goal quite well; and since the goal is to improve locality of reference, not 'make perfect' or such, no alert is required when we spill over. - Allen S. Rout
Re: Collocation anomaly report
Hi Grant, I used to track collocation group spill overs when my servers were version 5 and used tapes. Now I am on v6 and almost all disk, so I don't do that anymore. Anyway, I used a mysql database on my desktop system. I would dump data from the tsm servers and load it into mysql where I could do manipulations not allowed in the tsm servers. Then I would run a report which showed among other things volumes which have data from more than 1 collocation group. The key bit of data from tsm is q nodedata * which provides almost all the same info as a select from volumeusage, but is much faster. I can send you a sample report if you are interested. Bill Colwell Draper Lab -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Grant Street Sent: Tuesday, April 16, 2013 7:40 PM To: ADSM-L@VM.MARIST.EDU Subject: Collocation anomaly report Hello We use collocation to segment data into collocation groups and nodes, but recently found that collocation is on a best efforts basis and will use any tape if there is not enough space. I understand the theory behind this but it does not help with compliance requirements. I know that we should make sure that there are always enough free tapes, but without any way to know we have no proof that we are in compliance. I have created an RFE https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfeCR_ID=33537 . Please vote if you agree:-) While I wait a more than two years for this to be implemented, I was wondering if anyone had a way to report on any Collocation anomalies? I created the following but still not complete enough select volume_name, count(volume_name) as Nodes_per_volume from (select Unique volume_name , volumeusage.node_name from volumeusage, nodes where nodes.node_name = volumeusage.node_name and nodes. collocgroup_name is null) group by (volume_name) having count (volume_name) 1 and select unique volume_name, count(volume_name) as Groups_per_volume from (select Unique volume_name , collocgroup_name from volumeusage, nodes where nodes.node_name = volumeusage.node_name ) group by (volume_name) having count(volume_name) 1 Thanks in advance Grant
Re: Collocation anomaly report
You might try this query: select cast(vu.stgpool_name as char(9)) as Pool, - (select access from volumes where volume_name=vu.volume_name) as Access, - cast(vu.volume_name as char(9)) as Volume, - cast(count(distinct nd.collocgroup_name) as decimal(4,0)) as Groups, - cast(count(distinct vu.node_name) as decimal(3,0)) as Nodes, - cast(count(distinct vu.filespace_name) as decimal(5,0)) as FSpaces, - (select pct_utilized from volumes where volume_name=vu.volume_name) as Util, - (select pct_reclaim from volumes where volume_name=vu.volume_name) as Recl - from volumeusage vu, nodes nd - where vu.node_name=nd.node_name - group by vu.stgpool_name, vu.volume_name - order by Groups desc, Nodes desc, FSpaces desc We assign all nodes to collocation groups, even if it is only one node. Good luck. Bob. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Colwell, William F. Sent: Wednesday, April 17, 2013 9:44 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Collocation anomaly report Hi Grant, I used to track collocation group spill overs when my servers were version 5 and used tapes. Now I am on v6 and almost all disk, so I don't do that anymore. Anyway, I used a mysql database on my desktop system. I would dump data from the tsm servers and load it into mysql where I could do manipulations not allowed in the tsm servers. Then I would run a report which showed among other things volumes which have data from more than 1 collocation group. The key bit of data from tsm is q nodedata * which provides almost all the same info as a select from volumeusage, but is much faster. I can send you a sample report if you are interested. Bill Colwell Draper Lab -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Grant Street Sent: Tuesday, April 16, 2013 7:40 PM To: ADSM-L@VM.MARIST.EDU Subject: Collocation anomaly report Hello We use collocation to segment data into collocation groups and nodes, but recently found that collocation is on a best efforts basis and will use any tape if there is not enough space. I understand the theory behind this but it does not help with compliance requirements. I know that we should make sure that there are always enough free tapes, but without any way to know we have no proof that we are in compliance. I have created an RFE https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfeCR_ID=33537 . Please vote if you agree:-) While I wait a more than two years for this to be implemented, I was wondering if anyone had a way to report on any Collocation anomalies? I created the following but still not complete enough select volume_name, count(volume_name) as Nodes_per_volume from (select Unique volume_name , volumeusage.node_name from volumeusage, nodes where nodes.node_name = volumeusage.node_name and nodes. collocgroup_name is null) group by (volume_name) having count (volume_name) 1 and select unique volume_name, count(volume_name) as Groups_per_volume from (select Unique volume_name , collocgroup_name from volumeusage, nodes where nodes.node_name = volumeusage.node_name ) group by (volume_name) having count(volume_name) 1 Thanks in advance Grant This electronic transmission and any documents accompanying this electronic transmission contain confidential information belonging to the sender. This information may be legally privileged. The information is intended only for the use of the individual or entity named above. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or the taking of any action in reliance on or regarding the contents of this electronically transmitted information is strictly prohibited.
Collocation anomaly report
Hello We use collocation to segment data into collocation groups and nodes, but recently found that collocation is on a best efforts basis and will use any tape if there is not enough space. I understand the theory behind this but it does not help with compliance requirements. I know that we should make sure that there are always enough free tapes, but without any way to know we have no proof that we are in compliance. I have created an RFE https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfeCR_ID=33537 . Please vote if you agree:-) While I wait a more than two years for this to be implemented, I was wondering if anyone had a way to report on any Collocation anomalies? I created the following but still not complete enough select volume_name, count(volume_name) as Nodes_per_volume from (select Unique volume_name , volumeusage.node_name from volumeusage, nodes where nodes.node_name = volumeusage.node_name and nodes. collocgroup_name is null) group by (volume_name) having count (volume_name) 1 and select unique volume_name, count(volume_name) as Groups_per_volume from (select Unique volume_name , collocgroup_name from volumeusage, nodes where nodes.node_name = volumeusage.node_name ) group by (volume_name) having count(volume_name) 1 Thanks in advance Grant
Re: Collocation anomaly report
If you absolutely need for nodes to be isolated on their own media, why aren't they in their own individual domains which point to their own storage pools, all of which might share a library? Frankly, I like that TSM will override collocation preferences when its at MAXSCR for volumes in a pool. Just a thought, Nick On Apr 16, 2013, at 6:40 PM, Grant Street gra...@al.com.au wrote: Hello We use collocation to segment data into collocation groups and nodes, but recently found that collocation is on a best efforts basis and will use any tape if there is not enough space. I understand the theory behind this but it does not help with compliance requirements. I know that we should make sure that there are always enough free tapes, but without any way to know we have no proof that we are in compliance. snip Grant
Re: Collocation anomaly report
I understand that getting a good backup is the most important, but when you offer a feature that is best efforts with no way to verify, why have the feature? eg TSM will use best efforts to do a backup of a client. As you know there are times when for one reason or another a backup cannot be done. Imagine now, TSM does not give you a way to report if the backup succeeded, if it generated an error or when it started the backup? That is what happens when it fails to collocate, there is no report, it does not generate an error, warning or information message nor does post a message to any log that it has had to resort to non collocation. Does that make sense? I will need to look at the implications of splitting it out based on domains, thanks for the heads up. Grant On 17/04/13 10:15, Nick Laflamme wrote: If you absolutely need for nodes to be isolated on their own media, why aren't they in their own individual domains which point to their own storage pools, all of which might share a library? Frankly, I like that TSM will override collocation preferences when its at MAXSCR for volumes in a pool. Just a thought, Nick On Apr 16, 2013, at 6:40 PM, Grant Street gra...@al.com.au wrote: Hello We use collocation to segment data into collocation groups and nodes, but recently found that collocation is on a best efforts basis and will use any tape if there is not enough space. I understand the theory behind this but it does not help with compliance requirements. I know that we should make sure that there are always enough free tapes, but without any way to know we have no proof that we are in compliance. snip Grant
Re: Collocation anomaly report
Grant, If you care to script, do daily: Select new volumes assigned to pools in last 24 hours. (Maybe search actlog for ANR1340I messages.) Filter for volumes in pools with collocate (NODE or GROUP). For each volume check volumeusage for any nodes that do not belong. Assume the first node belongs. Report those volumes as candidates for move nodedata. Richard -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Grant Street Sent: Tuesday, April 16, 2013 7:40 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Collocation anomaly report Hello We use collocation to segment data into collocation groups and nodes, but recently found that collocation is on a best efforts basis and will use any tape if there is not enough space. I understand the theory behind this but it does not help with compliance requirements. I know that we should make sure that there are always enough free tapes, but without any way to know we have no proof that we are in compliance. I have created an RFE https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfeCR_ID=33537 . Please vote if you agree:-) While I wait a more than two years for this to be implemented, I was wondering if anyone had a way to report on any Collocation anomalies? I created the following but still not complete enough select volume_name, count(volume_name) as Nodes_per_volume from (select Unique volume_name , volumeusage.node_name from volumeusage, nodes where nodes.node_name = volumeusage.node_name and nodes. collocgroup_name is null) group by (volume_name) having count (volume_name) 1 and select unique volume_name, count(volume_name) as Groups_per_volume from (select Unique volume_name , collocgroup_name from volumeusage, nodes where nodes.node_name = volumeusage.node_name ) group by (volume_name) having count(volume_name) 1 Thanks in advance Grant
