Re: Reclamation is not happening - HELP!
This slowness can happen when reclamation is done from primary storage pool tapes also. Assume content of the offsite tape is scattered over multiple on site tapes and tape drive has to seek to multiple location on tape to pick up required piece of data can add up to significant tape loading/unloading and tape seek operation. So it is not unusual for an old offsite tape to take long time for reclamation. You should actually allow the reclamation process to finish. If you feel that reclamation process is keeping your tape drives engaged for long, then you can stop reclamation of offsite tapes alltogather and allow the offsite tapes to become empty due to natural expiration process. Samiran Das Rushforth, Tim [EMAIL PROTECTED]To: [EMAIL PROTECTED] PEG.MB.CA cc: Sent by: ADSM: DistSubject: Re: Reclamation is not happening - Stor Manager HELP! [EMAIL PROTECTED] U 05/22/2002 01:13 AM Please respond to ADSM: Dist Stor Manager Offsite reclamation runs very slow when processing input data from disk. (see http://msgs.adsm.org/cgi-bin/get/adsm0203/501.html) Also with offsite reclamation, TSM will process all the eligible offsite tapes at once (it does not process one offsite tape then the next). Have you tried a higher value, eg how many tapes are eligible over 95%, try this value etc. Tim Rushforth City of Winnipeg -Original Message- From: Coats, Jack [mailto:[EMAIL PROTECTED]] Sent: Tuesday, May 21, 2002 2:25 PM To: [EMAIL PROTECTED] Subject: Re: Reclamation is not happening - HELP! Currently, I have done the following: Allowed expiration to complete fully. Started reclamation again. It is running now but very slowly In the last 4+ hours, moved 55K files and 1.45G of data (from q proc) or if I do a query volume * access=readw,reado status=full,filling stgpool=copypool it says my LTO is 17% full. The machine does nothing but TSM, and is not under a load from I/O, networking, virtual memory, storage, or processor as far as I can tell. In any case it isn't filling quickly at all. Suggestions? PS... Thanks for all the comments suggestions sofar! ... Jack
Re: Reclamation is not happening - HELP!
Offsite reclamation runs very slow when processing input data from disk. (see http://msgs.adsm.org/cgi-bin/get/adsm0203/501.html) Also with offsite reclamation, TSM will process all the eligible offsite tapes at once (it does not process one offsite tape then the next). Have you tried a higher value, eg how many tapes are eligible over 95%, try this value etc. Tim Rushforth City of Winnipeg -Original Message- From: Coats, Jack [mailto:[EMAIL PROTECTED]] Sent: Tuesday, May 21, 2002 2:25 PM To: [EMAIL PROTECTED] Subject: Re: Reclamation is not happening - HELP! Currently, I have done the following: Allowed expiration to complete fully. Started reclamation again. It is running now but very slowly In the last 4+ hours, moved 55K files and 1.45G of data (from q proc) or if I do a query volume * access=readw,reado status=full,filling stgpool=copypool it says my LTO is 17% full. The machine does nothing but TSM, and is not under a load from I/O, networking, virtual memory, storage, or processor as far as I can tell. In any case it isn't filling quickly at all. Suggestions? PS... Thanks for all the comments suggestions sofar! ... Jack
Re: Reclamation is not happening - HELP!
Currently, I have done the following: Allowed expiration to complete fully. Started reclamation again. It is running now but very slowly In the last 4+ hours, moved 55K files and 1.45G of data (from q proc) or if I do a query volume * access=readw,reado status=full,filling stgpool=copypool it says my LTO is 17% full. The machine does nothing but TSM, and is not under a load from I/O, networking, virtual memory, storage, or processor as far as I can tell. In any case it isn't filling quickly at all. Suggestions? PS... Thanks for all the comments suggestions sofar! ... Jack -Original Message- From: Roger Deschner [mailto:[EMAIL PROTECTED]] Sent: Sunday, May 19, 2002 12:02 AM To: [EMAIL PROTECTED] Subject: Re: Reclamation is not happening - HELP! This happened to me once. After pulling what remains of my hair out, I tried simply restarting the server. That fixed it. Also remember that the server checks the list of volumes to see if they can be reclaimed only about once an hour. (It's costly to calculate.) So any changes you make to the threshold level might not cause reclamation to start until as much as an hour later. Roger Deschner University of Illinois at Chicago [EMAIL PROTECTED] On Sat, 18 May 2002, Zlatko Krastev wrote: AFAIK during backup files are not 'marked' at all. They are just stored and during reclamation each file is compared to rules defined in its mgmtclass copygroup. That why reclamation takes so long. If you backup a file and according to *current* rules a version should expire, later update copygroup to retain more versions (or for longer period) and just afterward run expiration - the file should not be expired. If it is marked during backup/archive it would expire which would be against ADSM/TSM ideology and should be treated as bug. Jack, sometimes it helps to change reclamation/migration threshold to 100 and minute or two later to lower it. I've digged deeper to analyse why. Zlatko Krastev IT Consultant Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] Sent by:ADSM: Dist Stor Manager [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: Subject:Re: Reclamation is not happening - HELP! Jack, As TSM ages backup and archive data, the files only get 'marked' for deletion. You must run an EXPIRE INVENTORY command to actually flip the switch from 'marked' for deletion to deleted. Then your storage pool reclamation should work. That is, unless I am missing something too!!! 8-) John G. Talafous IS Technical Principal The Timken CompanyGlobal Software Support P.O. Box 6927 Data Management 1835 Dueber Ave. S.W. Phone: (330)-471-3390 Canton, Ohio USA 44706-0927 Fax : (330)-471-4034 [EMAIL PROTECTED] http://www.timken.com -Original Message- From: Coats, Jack [mailto:[EMAIL PROTECTED]] Sent: Friday, May 17, 2002 1:50 PM To: [EMAIL PROTECTED] Subject: Reclamation is not happening - HELP! My tape use is going up, but nothing is being reclaimed from my copy pool. I must have done something wrong, but where do I go to look for whatever is wrong first? What am I missing? I can find nothing in the activity log, and even when I do a update stg copypool rec=5 and it should recover a lot, nothing happens. Sometimes the process starts up, but will sit for hours and nothing gets reclaimed. Running TSM 4.1.5 on NT, with 3583 library and LTO drives. It has been working, but stopped earlier this week. ... TIA ... Jack
Re: Reclamation is not happening - HELP! - Sorry
Bill, my fault. Thinking about many things wrote the wrong word (twice). I've read somewhere an explanation of this: thinking is much faster than speaking and only every fifth or tenth word goes out, so the results are looking silly (as I am). Sorry guys, I wanted to write expiration but wrote reclamation and misguided. On *expiration* files are compared to copygroup rules and *expiration* can take long for large number of files. Reclamation also can take long but it depends on many storage aspects and may vary. Zlatko Krastev IT Consultant Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] Sent by:ADSM: Dist Stor Manager [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: Subject:Re: Reclamation is not happening - HELP! Not RECLAMATION, but EXPIRATION processing where the management class retention policies are applied to all the version of a file and extra or old versions are expired. TSM keeps track of how much data remains on a tape volume after expiration and reports this my the Pct. Reclaimable Space is a Q VOL xxx F=D output. When you set the storage pool to RECLAIM=something less than 100% TSM compares each tape volume in the storage pool with this value. Volumes that have a Pct. reclaimable Space = this value are reclaimed. Expiration processing is done automatically by setting the EXPINTERVAL in the server options file, or setting it to 0 and running it manually with the EXPIRE INVENTORY command. Offsite reclamation takes so long because TSM is recreating the offsite volume from onsite copies of the files. SO you may end up with several tape mounts, tape prositioning just to read a single file. Onsite reclamation takes the volume to be reclaimed as input, copies all the good data from it to another onsite tape volume. Bill Boyer DSS, Inc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Zlatko Krastev Sent: Saturday, May 18, 2002 6:23 AM To: [EMAIL PROTECTED] Subject: Re: Reclamation is not happening - HELP! AFAIK during backup files are not 'marked' at all. They are just stored and during reclamation each file is compared to rules defined in its mgmtclass copygroup. That why reclamation takes so long. If you backup a file and according to *current* rules a version should expire, later update copygroup to retain more versions (or for longer period) and just afterward run expiration - the file should not be expired. If it is marked during backup/archive it would expire which would be against ADSM/TSM ideology and should be treated as bug. Jack, sometimes it helps to change reclamation/migration threshold to 100 and minute or two later to lower it. I've digged deeper to analyse why. Zlatko Krastev IT Consultant Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] Sent by:ADSM: Dist Stor Manager [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: Subject:Re: Reclamation is not happening - HELP! Jack, As TSM ages backup and archive data, the files only get 'marked' for deletion. You must run an EXPIRE INVENTORY command to actually flip the switch from 'marked' for deletion to deleted. Then your storage pool reclamation should work. That is, unless I am missing something too!!! 8-) John G. Talafous IS Technical Principal The Timken CompanyGlobal Software Support P.O. Box 6927 Data Management 1835 Dueber Ave. S.W. Phone: (330)-471-3390 Canton, Ohio USA 44706-0927 Fax : (330)-471-4034 [EMAIL PROTECTED] http://www.timken.com -Original Message- From: Coats, Jack [mailto:[EMAIL PROTECTED]] Sent: Friday, May 17, 2002 1:50 PM To: [EMAIL PROTECTED] Subject: Reclamation is not happening - HELP! My tape use is going up, but nothing is being reclaimed from my copy pool. I must have done something wrong, but where do I go to look for whatever is wrong first? What am I missing? I can find nothing in the activity log, and even when I do a update stg copypool rec=5 and it should recover a lot, nothing happens. Sometimes the process starts up, but will sit for hours and nothing gets reclaimed. Running TSM 4.1.5 on NT, with 3583 library and LTO drives. It has been working, but stopped earlier this week. ... TIA ... Jack
Re: Reclamation is not happening - HELP!
Roger, I am not on a stable ground on this issue but always thought that reclamation threshold is compared to result set by expiration in VOLUMES table. So calculation is performed only once (during expiration - now I am careful which word am using :-). And later comparision is not expensive - you cannot have millions of volumes so the table is not big and is easy to traverse. And I think both migration and reclamation are fired same way and retried in a minute. Just an opinion. Zlatko Krastev IT Consultant Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] Sent by:ADSM: Dist Stor Manager [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: Subject:Re: Reclamation is not happening - HELP! This happened to me once. After pulling what remains of my hair out, I tried simply restarting the server. That fixed it. Also remember that the server checks the list of volumes to see if they can be reclaimed only about once an hour. (It's costly to calculate.) So any changes you make to the threshold level might not cause reclamation to start until as much as an hour later. Roger Deschner University of Illinois at Chicago [EMAIL PROTECTED] On Sat, 18 May 2002, Zlatko Krastev wrote: AFAIK during backup files are not 'marked' at all. They are just stored and during reclamation each file is compared to rules defined in its mgmtclass copygroup. That why reclamation takes so long. If you backup a file and according to *current* rules a version should expire, later update copygroup to retain more versions (or for longer period) and just afterward run expiration - the file should not be expired. If it is marked during backup/archive it would expire which would be against ADSM/TSM ideology and should be treated as bug. Jack, sometimes it helps to change reclamation/migration threshold to 100 and minute or two later to lower it. I've digged deeper to analyse why. Zlatko Krastev IT Consultant Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] Sent by:ADSM: Dist Stor Manager [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: Subject:Re: Reclamation is not happening - HELP! Jack, As TSM ages backup and archive data, the files only get 'marked' for deletion. You must run an EXPIRE INVENTORY command to actually flip the switch from 'marked' for deletion to deleted. Then your storage pool reclamation should work. That is, unless I am missing something too!!! 8-) John G. Talafous IS Technical Principal The Timken CompanyGlobal Software Support P.O. Box 6927 Data Management 1835 Dueber Ave. S.W. Phone: (330)-471-3390 Canton, Ohio USA 44706-0927 Fax : (330)-471-4034 [EMAIL PROTECTED] http://www.timken.com -Original Message- From: Coats, Jack [mailto:[EMAIL PROTECTED]] Sent: Friday, May 17, 2002 1:50 PM To: [EMAIL PROTECTED] Subject: Reclamation is not happening - HELP! My tape use is going up, but nothing is being reclaimed from my copy pool. I must have done something wrong, but where do I go to look for whatever is wrong first? What am I missing? I can find nothing in the activity log, and even when I do a update stg copypool rec=5 and it should recover a lot, nothing happens. Sometimes the process starts up, but will sit for hours and nothing gets reclaimed. Running TSM 4.1.5 on NT, with 3583 library and LTO drives. It has been working, but stopped earlier this week. ... TIA ... Jack
Re: Reclamation is not happening - HELP!
This happened to me once. After pulling what remains of my hair out, I tried simply restarting the server. That fixed it. Also remember that the server checks the list of volumes to see if they can be reclaimed only about once an hour. (It's costly to calculate.) So any changes you make to the threshold level might not cause reclamation to start until as much as an hour later. Roger Deschner University of Illinois at Chicago [EMAIL PROTECTED] On Sat, 18 May 2002, Zlatko Krastev wrote: AFAIK during backup files are not 'marked' at all. They are just stored and during reclamation each file is compared to rules defined in its mgmtclass copygroup. That why reclamation takes so long. If you backup a file and according to *current* rules a version should expire, later update copygroup to retain more versions (or for longer period) and just afterward run expiration - the file should not be expired. If it is marked during backup/archive it would expire which would be against ADSM/TSM ideology and should be treated as bug. Jack, sometimes it helps to change reclamation/migration threshold to 100 and minute or two later to lower it. I've digged deeper to analyse why. Zlatko Krastev IT Consultant Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] Sent by:ADSM: Dist Stor Manager [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: Subject:Re: Reclamation is not happening - HELP! Jack, As TSM ages backup and archive data, the files only get 'marked' for deletion. You must run an EXPIRE INVENTORY command to actually flip the switch from 'marked' for deletion to deleted. Then your storage pool reclamation should work. That is, unless I am missing something too!!! 8-) John G. Talafous IS Technical Principal The Timken CompanyGlobal Software Support P.O. Box 6927 Data Management 1835 Dueber Ave. S.W. Phone: (330)-471-3390 Canton, Ohio USA 44706-0927 Fax : (330)-471-4034 [EMAIL PROTECTED] http://www.timken.com -Original Message- From: Coats, Jack [mailto:[EMAIL PROTECTED]] Sent: Friday, May 17, 2002 1:50 PM To: [EMAIL PROTECTED] Subject: Reclamation is not happening - HELP! My tape use is going up, but nothing is being reclaimed from my copy pool. I must have done something wrong, but where do I go to look for whatever is wrong first? What am I missing? I can find nothing in the activity log, and even when I do a update stg copypool rec=5 and it should recover a lot, nothing happens. Sometimes the process starts up, but will sit for hours and nothing gets reclaimed. Running TSM 4.1.5 on NT, with 3583 library and LTO drives. It has been working, but stopped earlier this week. ... TIA ... Jack
Re: Reclamation is not happening - HELP!
AFAIK during backup files are not 'marked' at all. They are just stored and during reclamation each file is compared to rules defined in its mgmtclass copygroup. That why reclamation takes so long. If you backup a file and according to *current* rules a version should expire, later update copygroup to retain more versions (or for longer period) and just afterward run expiration - the file should not be expired. If it is marked during backup/archive it would expire which would be against ADSM/TSM ideology and should be treated as bug. Jack, sometimes it helps to change reclamation/migration threshold to 100 and minute or two later to lower it. I've digged deeper to analyse why. Zlatko Krastev IT Consultant Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] Sent by:ADSM: Dist Stor Manager [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: Subject:Re: Reclamation is not happening - HELP! Jack, As TSM ages backup and archive data, the files only get 'marked' for deletion. You must run an EXPIRE INVENTORY command to actually flip the switch from 'marked' for deletion to deleted. Then your storage pool reclamation should work. That is, unless I am missing something too!!! 8-) John G. Talafous IS Technical Principal The Timken CompanyGlobal Software Support P.O. Box 6927 Data Management 1835 Dueber Ave. S.W. Phone: (330)-471-3390 Canton, Ohio USA 44706-0927 Fax : (330)-471-4034 [EMAIL PROTECTED] http://www.timken.com -Original Message- From: Coats, Jack [mailto:[EMAIL PROTECTED]] Sent: Friday, May 17, 2002 1:50 PM To: [EMAIL PROTECTED] Subject: Reclamation is not happening - HELP! My tape use is going up, but nothing is being reclaimed from my copy pool. I must have done something wrong, but where do I go to look for whatever is wrong first? What am I missing? I can find nothing in the activity log, and even when I do a update stg copypool rec=5 and it should recover a lot, nothing happens. Sometimes the process starts up, but will sit for hours and nothing gets reclaimed. Running TSM 4.1.5 on NT, with 3583 library and LTO drives. It has been working, but stopped earlier this week. ... TIA ... Jack
Re: Reclamation is not happening - HELP!
Not RECLAMATION, but EXPIRATION processing where the management class retention policies are applied to all the version of a file and extra or old versions are expired. TSM keeps track of how much data remains on a tape volume after expiration and reports this my the Pct. Reclaimable Space is a Q VOL xxx F=D output. When you set the storage pool to RECLAIM=something less than 100% TSM compares each tape volume in the storage pool with this value. Volumes that have a Pct. reclaimable Space = this value are reclaimed. Expiration processing is done automatically by setting the EXPINTERVAL in the server options file, or setting it to 0 and running it manually with the EXPIRE INVENTORY command. Offsite reclamation takes so long because TSM is recreating the offsite volume from onsite copies of the files. SO you may end up with several tape mounts, tape prositioning just to read a single file. Onsite reclamation takes the volume to be reclaimed as input, copies all the good data from it to another onsite tape volume. Bill Boyer DSS, Inc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Zlatko Krastev Sent: Saturday, May 18, 2002 6:23 AM To: [EMAIL PROTECTED] Subject: Re: Reclamation is not happening - HELP! AFAIK during backup files are not 'marked' at all. They are just stored and during reclamation each file is compared to rules defined in its mgmtclass copygroup. That why reclamation takes so long. If you backup a file and according to *current* rules a version should expire, later update copygroup to retain more versions (or for longer period) and just afterward run expiration - the file should not be expired. If it is marked during backup/archive it would expire which would be against ADSM/TSM ideology and should be treated as bug. Jack, sometimes it helps to change reclamation/migration threshold to 100 and minute or two later to lower it. I've digged deeper to analyse why. Zlatko Krastev IT Consultant Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] Sent by:ADSM: Dist Stor Manager [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: Subject:Re: Reclamation is not happening - HELP! Jack, As TSM ages backup and archive data, the files only get 'marked' for deletion. You must run an EXPIRE INVENTORY command to actually flip the switch from 'marked' for deletion to deleted. Then your storage pool reclamation should work. That is, unless I am missing something too!!! 8-) John G. Talafous IS Technical Principal The Timken CompanyGlobal Software Support P.O. Box 6927 Data Management 1835 Dueber Ave. S.W. Phone: (330)-471-3390 Canton, Ohio USA 44706-0927 Fax : (330)-471-4034 [EMAIL PROTECTED] http://www.timken.com -Original Message- From: Coats, Jack [mailto:[EMAIL PROTECTED]] Sent: Friday, May 17, 2002 1:50 PM To: [EMAIL PROTECTED] Subject: Reclamation is not happening - HELP! My tape use is going up, but nothing is being reclaimed from my copy pool. I must have done something wrong, but where do I go to look for whatever is wrong first? What am I missing? I can find nothing in the activity log, and even when I do a update stg copypool rec=5 and it should recover a lot, nothing happens. Sometimes the process starts up, but will sit for hours and nothing gets reclaimed. Running TSM 4.1.5 on NT, with 3583 library and LTO drives. It has been working, but stopped earlier this week. ... TIA ... Jack
Reclamation is not happening - HELP!
My tape use is going up, but nothing is being reclaimed from my copy pool. I must have done something wrong, but where do I go to look for whatever is wrong first? What am I missing? I can find nothing in the activity log, and even when I do a update stg copypool rec=5 and it should recover a lot, nothing happens. Sometimes the process starts up, but will sit for hours and nothing gets reclaimed. Running TSM 4.1.5 on NT, with 3583 library and LTO drives. It has been working, but stopped earlier this week. ... TIA ... Jack
Re: Reclamation is not happening - HELP!
Jack, As TSM ages backup and archive data, the files only get 'marked' for deletion. You must run an EXPIRE INVENTORY command to actually flip the switch from 'marked' for deletion to deleted. Then your storage pool reclamation should work. That is, unless I am missing something too!!! 8-) John G. Talafous IS Technical Principal The Timken CompanyGlobal Software Support P.O. Box 6927 Data Management 1835 Dueber Ave. S.W. Phone: (330)-471-3390 Canton, Ohio USA 44706-0927 Fax : (330)-471-4034 [EMAIL PROTECTED] http://www.timken.com -Original Message- From: Coats, Jack [mailto:[EMAIL PROTECTED]] Sent: Friday, May 17, 2002 1:50 PM To: [EMAIL PROTECTED] Subject: Reclamation is not happening - HELP! My tape use is going up, but nothing is being reclaimed from my copy pool. I must have done something wrong, but where do I go to look for whatever is wrong first? What am I missing? I can find nothing in the activity log, and even when I do a update stg copypool rec=5 and it should recover a lot, nothing happens. Sometimes the process starts up, but will sit for hours and nothing gets reclaimed. Running TSM 4.1.5 on NT, with 3583 library and LTO drives. It has been working, but stopped earlier this week. ... TIA ... Jack
