Re: Where is the missing 38GB?
According to this I have around 90GB of partially empty pages in this database. Two years ago it was at 60% utilization. After several large filespace deletions it has not recovered any pages in the DB for reuse. If I reduce the DB by the 1,012 MB of free space it will fail, reporting various SQL Table space errors. tsm: TSMS01q db f=d Available Space (MB): 120,000 Assigned Capacity (MB): 97,136 Maximum Extension (MB): 22,864 Maximum Reduction (MB): 1,012 Page Size (bytes): 4,096 Total Usable Pages: 24,866,816 Used Pages: 515,955 Pct Util: 2.1 Max. Pct Util: 3.7 Physical Volumes: 24 Buffer Pool Pages: 175,000 Total Buffer Requests: 2,271,606,552 Cache Hit Pct.: 99.97 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 0 Changed Since Last Backup (MB): 666.78 Percentage Changed: 33.08 Last Complete Backup Date/Time: 12/08/2008 06:01:48 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): From: Remco Post [mailto:[EMAIL PROTECTED] Sent: Fri 05/12/2008 23:19 To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? On Dec 5, 2008, at 16:55 , Phillip Burgess wrote: From my experience, TSM is not capable of re-using the space once it becomes fragmented. TSM will reuse empty pages, it will not reuse empty space in a partially used page, just like tapes, the cycle is empty-filling- full-empty except that there is no page-reclaim process. I have worked on servers with 150GB databases reduced down to 50% utilization after several large filespace deletions. We could not recover any space on the database and it continued to fill until it eventually failed to run a select statement, stating that there was not enough temporary space in the DB to process the statement. The DB utilization was still at 52%. A reload of the database reduced the size down to 75GB, we have been using it for nearly a year without any consequences from the reload. I have seen databases with fragmentation as high as 95%, no option left other than to export or reload the database Phil From: Roger Deschner [mailto:[EMAIL PROTECTED] Sent: Thu 04/12/2008 22:41 To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? . The missing 38GB is just simply gone. Wave it goodbye. Disks are cheap - find something more expensive to worry about. You will get it back, though, if you ever refill this database with data. But wait, you say, won't the new data be fragmented too? Sure, just like it would be if you started fresh with an empty database and let it run a year or so. Our 377gb database is from 1999 and has never been audited fully or reloaded, and is running smoothly on v5.5.1 now. We have done some large DELETE FILESPACE operations as we move some nodes to a second server, and some space has disappeared just like you report, but I'm not really worried about it. It will get reused by natural growth. OTOH, judging by the amount of data left in your database, an unload/load cycle should be cheap and fast, so why not? At least try it as a test, reloading to a test server, and let us know what happened. We're all waiting anxiously to hear - because we on this list can argue about TSM database fragmentation forever, as you have just seen. A third idea was suggested already - DELETE DBVOL. I've watched this remarkable command work, and it's like reclamation, except on the database. It's going to run for a very long time, and it's goal is not defragmentation so it won't do very well at that, but it will accomplish a basic reclamation operation on this database. The nice thing about DELETE DBVOL is that it can run with the system up and running. The not-so-nice thing about it is that it's unmirrored. You've got to delete all the mirror copies before DELETE DBVOL can work its real magic, and while it does you're very badly exposed to a single-disk failure, especially because it's slow. Therefore, before using DELETE DBVOL for this kind of thing, I always move that database extent to something that does hardware mirroring such as a commonplace hardware RAID box, or SSA RAID, etc. Roger Deschner University of Illinois at Chicago [EMAIL PROTECTED] Whether you think you can or can not, you are right. On Tue, 2 Dec 2008, Remco Post wrote: I'd say, from what I read in the thread, that an unload/load at this point will remove a lot of fragmentation, even though the estimate
Re: Where is the missing 38GB?
Unfortunately, it does seem to be some kind of design issue. I tried the create new DBVOL - delete OLD and basically shuffled all 5-DBvolumes, to no avail. The numbers remain the same after ending up with the same amount of DB space/volumes. In this case, disk is not cheap since it is an old test machine with a finite fixed internal disk that will not be expanded. I am stuck with what I have. Since the DB stats keep telling me I have data in 40GB of DB volumes that I can not release/reclaim, I am stuck with what I have. I wonder if I send enough data to it to really fill up the DB, would that resolve the bad DB pointer or would it just quit when it fills up to it's imaginary highwater mark? Yeah, I could do the reloadI could also just delete and start all over again since it is a TEST system. Nothing of any value in the DB. This is just nawing at me as to why there isn't a way to resolve it without extreme measures (how long would it take for you to unload/reload your 377GB DB?). Where is the internal tool that does a full/real AUDITDB that repairs such problems? Roger Deschner [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/04/2008 05:45 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? . The missing 38GB is just simply gone. Wave it goodbye. Disks are cheap - find something more expensive to worry about. You will get it back, though, if you ever refill this database with data. But wait, you say, won't the new data be fragmented too? Sure, just like it would be if you started fresh with an empty database and let it run a year or so. Our 377gb database is from 1999 and has never been audited fully or reloaded, and is running smoothly on v5.5.1 now. We have done some large DELETE FILESPACE operations as we move some nodes to a second server, and some space has disappeared just like you report, but I'm not really worried about it. It will get reused by natural growth. OTOH, judging by the amount of data left in your database, an unload/load cycle should be cheap and fast, so why not? At least try it as a test, reloading to a test server, and let us know what happened. We're all waiting anxiously to hear - because we on this list can argue about TSM database fragmentation forever, as you have just seen. A third idea was suggested already - DELETE DBVOL. I've watched this remarkable command work, and it's like reclamation, except on the database. It's going to run for a very long time, and it's goal is not defragmentation so it won't do very well at that, but it will accomplish a basic reclamation operation on this database. The nice thing about DELETE DBVOL is that it can run with the system up and running. The not-so-nice thing about it is that it's unmirrored. You've got to delete all the mirror copies before DELETE DBVOL can work its real magic, and while it does you're very badly exposed to a single-disk failure, especially because it's slow. Therefore, before using DELETE DBVOL for this kind of thing, I always move that database extent to something that does hardware mirroring such as a commonplace hardware RAID box, or SSA RAID, etc. Roger Deschner University of Illinois at Chicago [EMAIL PROTECTED] Whether you think you can or can not, you are right. On Tue, 2 Dec 2008, Remco Post wrote: I'd say, from what I read in the thread, that an unload/load at this point will remove a lot of fragmentation, even though the estimate says nill. The other option is to run an export server, reformat everything, and then import server. You'll probably want to save your devconf.txt so you can easily recreate the stgpools, devclasses and server2server comms you might have. On Dec 2, 2008, at 18:20 , Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time:
Re: Where is the missing 38GB?
From my experience, TSM is not capable of re-using the space once it becomes fragmented. I have worked on servers with 150GB databases reduced down to 50% utilization after several large filespace deletions. We could not recover any space on the database and it continued to fill until it eventually failed to run a select statement, stating that there was not enough temporary space in the DB to process the statement. The DB utilization was still at 52%. A reload of the database reduced the size down to 75GB, we have been using it for nearly a year without any consequences from the reload. I have seen databases with fragmentation as high as 95%, no option left other than to export or reload the database Phil From: Roger Deschner [mailto:[EMAIL PROTECTED] Sent: Thu 04/12/2008 22:41 To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? . The missing 38GB is just simply gone. Wave it goodbye. Disks are cheap - find something more expensive to worry about. You will get it back, though, if you ever refill this database with data. But wait, you say, won't the new data be fragmented too? Sure, just like it would be if you started fresh with an empty database and let it run a year or so. Our 377gb database is from 1999 and has never been audited fully or reloaded, and is running smoothly on v5.5.1 now. We have done some large DELETE FILESPACE operations as we move some nodes to a second server, and some space has disappeared just like you report, but I'm not really worried about it. It will get reused by natural growth. OTOH, judging by the amount of data left in your database, an unload/load cycle should be cheap and fast, so why not? At least try it as a test, reloading to a test server, and let us know what happened. We're all waiting anxiously to hear - because we on this list can argue about TSM database fragmentation forever, as you have just seen. A third idea was suggested already - DELETE DBVOL. I've watched this remarkable command work, and it's like reclamation, except on the database. It's going to run for a very long time, and it's goal is not defragmentation so it won't do very well at that, but it will accomplish a basic reclamation operation on this database. The nice thing about DELETE DBVOL is that it can run with the system up and running. The not-so-nice thing about it is that it's unmirrored. You've got to delete all the mirror copies before DELETE DBVOL can work its real magic, and while it does you're very badly exposed to a single-disk failure, especially because it's slow. Therefore, before using DELETE DBVOL for this kind of thing, I always move that database extent to something that does hardware mirroring such as a commonplace hardware RAID box, or SSA RAID, etc. Roger Deschner University of Illinois at Chicago [EMAIL PROTECTED] Whether you think you can or can not, you are right. On Tue, 2 Dec 2008, Remco Post wrote: I'd say, from what I read in the thread, that an unload/load at this point will remove a lot of fragmentation, even though the estimate says nill. The other option is to run an export server, reformat everything, and then import server. You'll probably want to save your devconf.txt so you can easily recreate the stgpools, devclasses and server2server comms you might have. On Dec 2, 2008, at 18:20 , Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: 12:15:11 PM TSMTEST : q stg Storage Device
Re: Where is the missing 38GB?
Kinda what I figured. However, with the upcoming V6.1 and conversion to DB2, I would suspect you need a very clean database for the conversion. I wonder if they are going to ever supply some tools to perform thorough scrubbing of the DB and resolve these kid of problems or just make everyone do unload/reloads before attempting a conversion. I don't think you want to have problems during the conversion to a different DB format Phillip Burgess [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/05/2008 10:56 AM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? From my experience, TSM is not capable of re-using the space once it becomes fragmented. I have worked on servers with 150GB databases reduced down to 50% utilization after several large filespace deletions. We could not recover any space on the database and it continued to fill until it eventually failed to run a select statement, stating that there was not enough temporary space in the DB to process the statement. The DB utilization was still at 52%. A reload of the database reduced the size down to 75GB, we have been using it for nearly a year without any consequences from the reload. I have seen databases with fragmentation as high as 95%, no option left other than to export or reload the database Phil From: Roger Deschner [mailto:[EMAIL PROTECTED] Sent: Thu 04/12/2008 22:41 To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? . The missing 38GB is just simply gone. Wave it goodbye. Disks are cheap - find something more expensive to worry about. You will get it back, though, if you ever refill this database with data. But wait, you say, won't the new data be fragmented too? Sure, just like it would be if you started fresh with an empty database and let it run a year or so. Our 377gb database is from 1999 and has never been audited fully or reloaded, and is running smoothly on v5.5.1 now. We have done some large DELETE FILESPACE operations as we move some nodes to a second server, and some space has disappeared just like you report, but I'm not really worried about it. It will get reused by natural growth. OTOH, judging by the amount of data left in your database, an unload/load cycle should be cheap and fast, so why not? At least try it as a test, reloading to a test server, and let us know what happened. We're all waiting anxiously to hear - because we on this list can argue about TSM database fragmentation forever, as you have just seen. A third idea was suggested already - DELETE DBVOL. I've watched this remarkable command work, and it's like reclamation, except on the database. It's going to run for a very long time, and it's goal is not defragmentation so it won't do very well at that, but it will accomplish a basic reclamation operation on this database. The nice thing about DELETE DBVOL is that it can run with the system up and running. The not-so-nice thing about it is that it's unmirrored. You've got to delete all the mirror copies before DELETE DBVOL can work its real magic, and while it does you're very badly exposed to a single-disk failure, especially because it's slow. Therefore, before using DELETE DBVOL for this kind of thing, I always move that database extent to something that does hardware mirroring such as a commonplace hardware RAID box, or SSA RAID, etc. Roger Deschner University of Illinois at Chicago [EMAIL PROTECTED] Whether you think you can or can not, you are right. On Tue, 2 Dec 2008, Remco Post wrote: I'd say, from what I read in the thread, that an unload/load at this point will remove a lot of fragmentation, even though the estimate says nill. The other option is to run an export server, reformat everything, and then import server. You'll probably want to save your devconf.txt so you can easily recreate the stgpools, devclasses and server2server comms you might have. On Dec 2, 2008, at 18:20 , Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00
Re: Where is the missing 38GB?
On Dec 5, 2008, at 16:55 , Phillip Burgess wrote: From my experience, TSM is not capable of re-using the space once it becomes fragmented. TSM will reuse empty pages, it will not reuse empty space in a partially used page, just like tapes, the cycle is empty-filling- full-empty except that there is no page-reclaim process. I have worked on servers with 150GB databases reduced down to 50% utilization after several large filespace deletions. We could not recover any space on the database and it continued to fill until it eventually failed to run a select statement, stating that there was not enough temporary space in the DB to process the statement. The DB utilization was still at 52%. A reload of the database reduced the size down to 75GB, we have been using it for nearly a year without any consequences from the reload. I have seen databases with fragmentation as high as 95%, no option left other than to export or reload the database Phil From: Roger Deschner [mailto:[EMAIL PROTECTED] Sent: Thu 04/12/2008 22:41 To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? . The missing 38GB is just simply gone. Wave it goodbye. Disks are cheap - find something more expensive to worry about. You will get it back, though, if you ever refill this database with data. But wait, you say, won't the new data be fragmented too? Sure, just like it would be if you started fresh with an empty database and let it run a year or so. Our 377gb database is from 1999 and has never been audited fully or reloaded, and is running smoothly on v5.5.1 now. We have done some large DELETE FILESPACE operations as we move some nodes to a second server, and some space has disappeared just like you report, but I'm not really worried about it. It will get reused by natural growth. OTOH, judging by the amount of data left in your database, an unload/load cycle should be cheap and fast, so why not? At least try it as a test, reloading to a test server, and let us know what happened. We're all waiting anxiously to hear - because we on this list can argue about TSM database fragmentation forever, as you have just seen. A third idea was suggested already - DELETE DBVOL. I've watched this remarkable command work, and it's like reclamation, except on the database. It's going to run for a very long time, and it's goal is not defragmentation so it won't do very well at that, but it will accomplish a basic reclamation operation on this database. The nice thing about DELETE DBVOL is that it can run with the system up and running. The not-so-nice thing about it is that it's unmirrored. You've got to delete all the mirror copies before DELETE DBVOL can work its real magic, and while it does you're very badly exposed to a single-disk failure, especially because it's slow. Therefore, before using DELETE DBVOL for this kind of thing, I always move that database extent to something that does hardware mirroring such as a commonplace hardware RAID box, or SSA RAID, etc. Roger Deschner University of Illinois at Chicago [EMAIL PROTECTED] Whether you think you can or can not, you are right. On Tue, 2 Dec 2008, Remco Post wrote: I'd say, from what I read in the thread, that an unload/load at this point will remove a lot of fragmentation, even though the estimate says nill. The other option is to run an export server, reformat everything, and then import server. You'll probably want to save your devconf.txt so you can easily recreate the stgpools, devclasses and server2server comms you might have. On Dec 2, 2008, at 18:20 , Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB
Re: Where is the missing 38GB?
. The missing 38GB is just simply gone. Wave it goodbye. Disks are cheap - find something more expensive to worry about. You will get it back, though, if you ever refill this database with data. But wait, you say, won't the new data be fragmented too? Sure, just like it would be if you started fresh with an empty database and let it run a year or so. Our 377gb database is from 1999 and has never been audited fully or reloaded, and is running smoothly on v5.5.1 now. We have done some large DELETE FILESPACE operations as we move some nodes to a second server, and some space has disappeared just like you report, but I'm not really worried about it. It will get reused by natural growth. OTOH, judging by the amount of data left in your database, an unload/load cycle should be cheap and fast, so why not? At least try it as a test, reloading to a test server, and let us know what happened. We're all waiting anxiously to hear - because we on this list can argue about TSM database fragmentation forever, as you have just seen. A third idea was suggested already - DELETE DBVOL. I've watched this remarkable command work, and it's like reclamation, except on the database. It's going to run for a very long time, and it's goal is not defragmentation so it won't do very well at that, but it will accomplish a basic reclamation operation on this database. The nice thing about DELETE DBVOL is that it can run with the system up and running. The not-so-nice thing about it is that it's unmirrored. You've got to delete all the mirror copies before DELETE DBVOL can work its real magic, and while it does you're very badly exposed to a single-disk failure, especially because it's slow. Therefore, before using DELETE DBVOL for this kind of thing, I always move that database extent to something that does hardware mirroring such as a commonplace hardware RAID box, or SSA RAID, etc. Roger Deschner University of Illinois at Chicago [EMAIL PROTECTED] Whether you think you can or can not, you are right. On Tue, 2 Dec 2008, Remco Post wrote: I'd say, from what I read in the thread, that an unload/load at this point will remove a lot of fragmentation, even though the estimate says nill. The other option is to run an export server, reformat everything, and then import server. You'll probably want to save your devconf.txt so you can easily recreate the stgpools, devclasses and server2server comms you might have. On Dec 2, 2008, at 18:20 , Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: 12:15:11 PM TSMTEST : q stg Storage Device EstimatedPctPct High Low Next Stora- Pool NameClass NameCapacity Util Migr Mig Mig ge Pool Pct Pct --- -- -- - - --- --- ARCHIVEPOOL DISK 0.0 M0.00.090 30 BACKUPPOOL DISK 123 G0.00.090 30 COPYPOOL-I- IBM3583-10.0 M0.0 BM3583-1 COPYPOOL-I- IBM3583-20.0 M0.0 BM3583-2 IBM3494-35- 3592E05 0.0 M0.00.090 70 92 I did a DSMSERV AUDITDB FIX=YES and the only thing it complained (and fixed) about was old schedules for non-existing nodes. Also did an EXPIRE INVENTORY. -- Met vriendelijke groeten, Remco Post [EMAIL PROTECTED] +31 6 248 21 622
Re: Where is the missing 38GB?
On Dec 4, 2008, at 23:41 , Roger Deschner wrote: . The missing 38GB is just simply gone. Wave it goodbye. Disks are cheap - find something more expensive to worry about. ok, I'll have to disagree here. an empty db (and this one is) should not have 100% fragmentation. And like my test server, with only 2*18GB internally, I'd hate to loose 38 ;-) You will get it back, though, if you ever refill this database with data. But wait, you say, won't the new data be fragmented too? Sure, just like it would be if you started fresh with an empty database and let it run a year or so. in normal operations, agreed, unless you agree that 100% fragmentation should at least clean up pretty nicely with an unload/load. Our 377gb database is from 1999 and has never been audited fully or reloaded, and is running smoothly on v5.5.1 now. We have done some large DELETE FILESPACE operations as we move some nodes to a second server, and some space has disappeared just like you report, but I'm not really worried about it. It will get reused by natural growth. Now, you should be worried. 377 GB is huge :) I agree, in a production db of that size, about 10-20% external fragmentation (free block amongst the used ones) is nothing to worry about, neither is 50% or more internal fragmentation (half used blocks). Zoltan's db is something completely different and a good example of a db that does apply for an unload/load cycle. OTOH, judging by the amount of data left in your database, an unload/load cycle should be cheap and fast, so why not? At least try it as a test, reloading to a test server, and let us know what happened. We're all waiting anxiously to hear - because we on this list can argue about TSM database fragmentation forever, as you have just seen. A third idea was suggested already - DELETE DBVOL. I've watched this remarkable command work, and it's like reclamation, except on the database. It's going to run for a very long time, and it's goal is not defragmentation so it won't do very well at that, but it will accomplish a basic reclamation operation on this database. The nice thing about DELETE DBVOL is that it can run with the system up and running. The not-so-nice thing about it is that it's unmirrored. You've got to delete all the mirror copies before DELETE DBVOL can work its real magic, and while it does you're very badly exposed to a single-disk failure, especially because it's slow. Therefore, before using DELETE DBVOL for this kind of thing, I always move that database extent to something that does hardware mirroring such as a commonplace hardware RAID box, or SSA RAID, etc. delete dbvol only solves external fragmentation, non-empty block will be moved as they are. In Zoltan's case, these make up about 100% of his database. I happen to like thinking and discussing about intresting TSM oddities, I even do it when I'm not at work. Zoltan, did you ever do the unload/load? Roger Deschner University of Illinois at Chicago [EMAIL PROTECTED] Whether you think you can or can not, you are right. On Tue, 2 Dec 2008, Remco Post wrote: I'd say, from what I read in the thread, that an unload/load at this point will remove a lot of fragmentation, even though the estimate says nill. The other option is to run an export server, reformat everything, and then import server. You'll probably want to save your devconf.txt so you can easily recreate the stgpools, devclasses and server2server comms you might have. On Dec 2, 2008, at 18:20 , Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined.
Re: Where is the missing 38GB?
Shoot me if I am wrong, but isn't this a correct result? The database uses 0,001 times the assigned capacity, that is around 50 Megabytes. Of these 50 Megabytes, zero MB is recoverable by a dbreorg. As of my knowledge estimate dbreorg operates on how much is actually utilized. So if you dump it and load it, it will consume the space(Pct.util times Assigned Capacity/100) minus the estimated recoverable space. The db will then fit onto something a little bigger than a 50 MB volume.(the output from the dump will tell you how much is needed). Sounds reasonable to me. Hans Chr. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Wednesday, December 03, 2008 3:09 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: Where is the missing 38GB? 9:08:28 AM TSMTEST : q db f=d Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,780 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 69,643 Cache Hit Pct.: 80.90 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 0 Changed Since Last Backup (MB): 0.38 Percentage Changed: 0.62 Last Complete Backup Date/Time: 12/02/2008 13:54:50 Estimate of Recoverable Space (MB): 0 Last Estimate of Recoverable Space (MB): 12/02/2008 12:41:54 Conway, Timothy [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/02/2008 05:43 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? How about a q db f=d now that the estimate dbreorgstats is finished? The q dbv f=d doesn't really give anything that's affected by the estimate. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Remco Post Sent: Tuesday, December 02, 2008 3:04 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? Ok, this is an almost empty database with a lot of fragmentation. I'd say, from what I read in the thread, that an unload/load at this point will remove a lot of fragmentation, even though the estimate says nill. The other option is to run an export server, reformat everything, and then import server. You'll probably want to save your devconf.txt so you can easily recreate the stgpools, devclasses and server2server comms you might have. On Dec 2, 2008, at 18:20 , Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: 12:15:11 PM TSMTEST : q stg Storage Device EstimatedPctPct High Low Next Stora- Pool NameClass NameCapacity Util Migr Mig Mig ge Pool Pct Pct --- -- -- - - --- --- ARCHIVEPOOL DISK 0.0 M0.00.090 30 BACKUPPOOL DISK 123 G0.00.090 30 COPYPOOL-I- IBM3583-10.0 M
Re: Where is the missing 38GB?
On 02/12, Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: ... I did a DSMSERV AUDITDB FIX=YES and the only thing it complained (and fixed) about was old schedules for non-existing nodes. Also did an EXPIRE INVENTORY. The DB is probably fragmented. Try: ESTIMATE DBREORGSTATS Q DBVOL F=D
Re: Where is the missing 38GB?
12/2/2008 12:41:46 PM ANR0984I Process 1 for ESTIMATE DBREORG started in the BACKGROUND at 12:41:46 PM. 12/2/2008 12:41:46 PM ANR1782W ESTIMATE DBREORG process 1 started - server performance may be degraded while this process is running. 12/2/2008 12:41:46 PM ANR0405I Session 78 ended for administrator ZFORRAY (WinNT). 12/2/2008 12:41:53 PM ANR1784I A database reorganization would reduce the database utilization by an estimated 0 MB. 12/2/2008 12:41:53 PM ANR0987I Process 1 for ESTIMATE DBREORG running in the BACKGROUND processed 1344 items with a completion state of SUCCESS at 12:41:54 PM. 12/2/2008 12:41:53 PM ANR0381I Buffer pool statistics were successfully reset. 12:42:33 PM TSMTEST : q dbvol f=d Volume Name (Copy 1): /opt/tivoli/tsm/server/bin/db.dsm Copy Status: Sync'd Volume Name (Copy 2): Copy Status: Undefined Volume Name (Copy 3): Copy Status: Undefined Available Space (MB): 16 Allocated Space (MB): 16 Free Space (MB): 0 Volume Name (Copy 1): /tsmpool/db/db2.dsm Copy Status: Sync'd Volume Name (Copy 2): Copy Status: Undefined Volume Name (Copy 3): Copy Status: Undefined Available Space (MB): 10,240 Allocated Space (MB): 10,240 Free Space (MB): 0 Volume Name (Copy 1): /tsmpool/db/db1.dsm Copy Status: Sync'd Volume Name (Copy 2): Copy Status: Undefined Volume Name (Copy 3): Copy Status: Undefined Available Space (MB): 10,240 Allocated Space (MB): 10,240 Free Space (MB): 0 Volume Name (Copy 1): /home/tsmdb/db3.dsm Copy Status: Sync'd Volume Name (Copy 2): Copy Status: Undefined Volume Name (Copy 3): Copy Status: Undefined Available Space (MB): 5,120 Allocated Space (MB): 5,120 Free Space (MB): 0 Volume Name (Copy 1): /tsmpool/db/db4.dsm Copy Status: Sync'd Volume Name (Copy 2): Copy Status: Undefined Volume Name (Copy 3): Copy Status: Undefined Available Space (MB): 10,240 Allocated Space (MB): 10,240 Free Space (MB): 0 Volume Name (Copy 1): /tsmpool/db/db5.dsm Copy Status: Sync'd Volume Name (Copy 2): Copy Status: Undefined Volume Name (Copy 3): Copy Status: Undefined Available Space (MB): 20,480 Allocated Space (MB): 17,408 Free Space (MB): 3,072 km [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/02/2008 12:40 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? On 02/12, Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: ... I did a DSMSERV AUDITDB FIX=YES and the only thing it complained (and fixed) about was old schedules for non-existing nodes. Also did an EXPIRE INVENTORY. The DB is probably fragmented. Try: ESTIMATE DBREORGSTATS Q DBVOL F=D
Re: Where is the missing 38GB?
The dreaded DSMSERV DUMPDB and DSMSERV LOADDB (not recommended) DSMSERV LOADDB (Reload the database) Use this command to reload a Tivoli Storage Manager database in optimal order. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 9:44 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: Where is the missing 38GB? 12/2/2008 12:41:46 PM ANR0984I Process 1 for ESTIMATE DBREORG started in the BACKGROUND at 12:41:46 PM. 12/2/2008 12:41:46 PM ANR1782W ESTIMATE DBREORG process 1 started - server performance may be degraded while this process is running. 12/2/2008 12:41:46 PM ANR0405I Session 78 ended for administrator ZFORRAY (WinNT). 12/2/2008 12:41:53 PM ANR1784I A database reorganization would reduce the database utilization by an estimated 0 MB. 12/2/2008 12:41:53 PM ANR0987I Process 1 for ESTIMATE DBREORG running in the BACKGROUND processed 1344 items with a completion state of SUCCESS at 12:41:54 PM. 12/2/2008 12:41:53 PM ANR0381I Buffer pool statistics were successfully reset. On 02/12, Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: ... I did a DSMSERV AUDITDB FIX=YES and the only thing it complained (and fixed) about was old schedules for non-existing nodes. Also did an EXPIRE INVENTORY. The DB is probably fragmented. Try: ESTIMATE DBREORGSTATS Q DBVOL F=D
Re: Where is the missing 38GB?
I'm waiting for the firestorm of comments following this suggestions. (Sitting back, waiting for fireworks.) -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Gee, Norman Sent: Tuesday, December 02, 2008 11:52 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? The dreaded DSMSERV DUMPDB and DSMSERV LOADDB (not recommended) DSMSERV LOADDB (Reload the database) Use this command to reload a Tivoli Storage Manager database in optimal order. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 9:44 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: Where is the missing 38GB? 12/2/2008 12:41:46 PM ANR0984I Process 1 for ESTIMATE DBREORG started in the BACKGROUND at 12:41:46 PM. 12/2/2008 12:41:46 PM ANR1782W ESTIMATE DBREORG process 1 started - server performance may be degraded while this process is running. 12/2/2008 12:41:46 PM ANR0405I Session 78 ended for administrator ZFORRAY (WinNT). 12/2/2008 12:41:53 PM ANR1784I A database reorganization would reduce the database utilization by an estimated 0 MB. 12/2/2008 12:41:53 PM ANR0987I Process 1 for ESTIMATE DBREORG running in the BACKGROUND processed 1344 items with a completion state of SUCCESS at 12:41:54 PM. 12/2/2008 12:41:53 PM ANR0381I Buffer pool statistics were successfully reset. On 02/12, Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: ... I did a DSMSERV AUDITDB FIX=YES and the only thing it complained (and fixed) about was old schedules for non-existing nodes. Also did an EXPIRE INVENTORY. The DB is probably fragmented. Try: ESTIMATE DBREORGSTATS Q DBVOL F=D
Re: Where is the missing 38GB?
Or better yet, I could just reformat everything, since this is a test system. I just want to know why.. Yeah, yeah, I realize V6.1 with DB2 will fix all DB issues.; Schneider, Jim [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/02/2008 12:58 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? I'm waiting for the firestorm of comments following this suggestions. (Sitting back, waiting for fireworks.) -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Gee, Norman Sent: Tuesday, December 02, 2008 11:52 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? The dreaded DSMSERV DUMPDB and DSMSERV LOADDB (not recommended) DSMSERV LOADDB (Reload the database) Use this command to reload a Tivoli Storage Manager database in optimal order. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 9:44 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: Where is the missing 38GB? 12/2/2008 12:41:46 PM ANR0984I Process 1 for ESTIMATE DBREORG started in the BACKGROUND at 12:41:46 PM. 12/2/2008 12:41:46 PM ANR1782W ESTIMATE DBREORG process 1 started - server performance may be degraded while this process is running. 12/2/2008 12:41:46 PM ANR0405I Session 78 ended for administrator ZFORRAY (WinNT). 12/2/2008 12:41:53 PM ANR1784I A database reorganization would reduce the database utilization by an estimated 0 MB. 12/2/2008 12:41:53 PM ANR0987I Process 1 for ESTIMATE DBREORG running in the BACKGROUND processed 1344 items with a completion state of SUCCESS at 12:41:54 PM. 12/2/2008 12:41:53 PM ANR0381I Buffer pool statistics were successfully reset. On 02/12, Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: ... I did a DSMSERV AUDITDB FIX=YES and the only thing it complained (and fixed) about was old schedules for non-existing nodes. Also did an EXPIRE INVENTORY. The DB is probably fragmented. Try: ESTIMATE DBREORGSTATS Q DBVOL F=D
Re: Where is the missing 38GB?
I would try doing a full DB backup. Can you send q dbv output too? [RC] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 9:20 AM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Where is the missing 38GB? I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: 12:15:11 PM TSMTEST : q stg Storage Device EstimatedPctPct High Low Next Stora- Pool NameClass NameCapacity Util Migr Mig Mig ge Pool Pct Pct --- -- -- - - --- --- ARCHIVEPOOL DISK 0.0 M0.00.090 30 BACKUPPOOL DISK 123 G0.00.090 30 COPYPOOL-I- IBM3583-10.0 M0.0 BM3583-1 COPYPOOL-I- IBM3583-20.0 M0.0 BM3583-2 IBM3494-35- 3592E05 0.0 M0.00.090 70 92 I did a DSMSERV AUDITDB FIX=YES and the only thing it complained (and fixed) about was old schedules for non-existing nodes. Also did an EXPIRE INVENTORY. DISCLAIMER: This message is intended for the sole use of the addressee, and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If you are not the addressee you are hereby notified that you may not use, copy, disclose, or distribute to anyone the message or any information contained in the message. If you have received this message in error, please immediately advise the sender by reply email and delete this message.
Re: Where is the missing 38GB?
If you have a habit of deleting filespaces on this test box, that would explain it. You'd be seeing the fragmentation that happens to the TSM db when such brute-force deletions are done. I had several customers that had similar issues with their production boxes. -- Mark Stapleton ([EMAIL PROTECTED]) CDW Berbee System engineer 7145 Boone Avenue North, Suite 140 Brooklyn Park MN 55428-1511 763-592-5963 www.berbee.com -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 12:02 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? Or better yet, I could just reformat everything, since this is a test system. I just want to know why.. Yeah, yeah, I realize V6.1 with DB2 will fix all DB issues.; Schneider, Jim [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/02/2008 12:58 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? I'm waiting for the firestorm of comments following this suggestions. (Sitting back, waiting for fireworks.) -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Gee, Norman Sent: Tuesday, December 02, 2008 11:52 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? The dreaded DSMSERV DUMPDB and DSMSERV LOADDB (not recommended) DSMSERV LOADDB (Reload the database) Use this command to reload a Tivoli Storage Manager database in optimal order. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 9:44 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: Where is the missing 38GB? 12/2/2008 12:41:46 PM ANR0984I Process 1 for ESTIMATE DBREORG started in the BACKGROUND at 12:41:46 PM. 12/2/2008 12:41:46 PM ANR1782W ESTIMATE DBREORG process 1 started - server performance may be degraded while this process is running. 12/2/2008 12:41:46 PM ANR0405I Session 78 ended for administrator ZFORRAY (WinNT). 12/2/2008 12:41:53 PM ANR1784I A database reorganization would reduce the database utilization by an estimated 0 MB. 12/2/2008 12:41:53 PM ANR0987I Process 1 for ESTIMATE DBREORG running in the BACKGROUND processed 1344 items with a completion state of SUCCESS at 12:41:54 PM. 12/2/2008 12:41:53 PM ANR0381I Buffer pool statistics were successfully reset. On 02/12, Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: ... I did a DSMSERV AUDITDB FIX=YES and the only thing it complained (and fixed) about was old schedules for non-existing nodes. Also did an EXPIRE INVENTORY. The DB is probably fragmented. Try: ESTIMATE DBREORGSTATS Q DBVOL F=D
Re: Where is the missing 38GB?
No change... 12/2/2008 1:53:43 PM ANR2017I Administrator ZFORRAY issued command: BACKUP DB dev=FILEDEVCLASS typ=Full scr=Yes 12/2/2008 1:54:49 PM ANR0984I Process 2 for DATABASE BACKUP started in the BACKGROUND at 01:54:50 PM. 12/2/2008 1:54:49 PM ANR2280I Full database backup started as process 2. 12/2/2008 1:54:49 PM ANR8340I FILE volume /tsmpool/filedirs/28244090.DBB mounted. 12/2/2008 1:54:49 PM ANR0513I Process 2 opened output volume /tsmpool/filedirs/28244090.DBB. 12/2/2008 1:54:49 PM ANR1360I Output volume /tsmpool/filedirs/28244090.DBB opened (sequence number 1). 12/2/2008 1:56:41 PM ANR4554I Backed up 34048 of 68561 database pages. 12/2/2008 1:57:11 PM ANR4554I Backed up 57024 of 68561 database pages. 12/2/2008 1:57:41 PM ANR4554I Backed up 61952 of 68561 database pages. 12/2/2008 1:58:11 PM ANR4554I Backed up 67392 of 68561 database pages. 12/2/2008 1:58:17 PM ANR1361I Output volume /tsmpool/filedirs/28244090.DBB closed. 12/2/2008 1:58:17 PM ANR0515I Process 2 closed volume /tsmpool/filedirs/28244090.DBB. 12/2/2008 1:58:17 PM ANR4550I Full database backup (process 2) complete, 68561 pages copied. 12/2/2008 1:58:17 PM ANR0985I Process 2 for DATABASE BACKUP running in the BACKGROUND completed with completion state SUCCESS at 01:58:18 PM. 1:59:53 PM TSMTEST : q db f=d Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,694 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 34,826 Cache Hit Pct.: 61.80 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 0 Changed Since Last Backup (MB): 0.04 Percentage Changed: 0.07 Last Complete Backup Date/Time: 12/02/2008 13:54:50 Estimate of Recoverable Space (MB): 0 Last Estimate of Recoverable Space (MB): 12/02/2008 12:41:54 --- 2:01:02 PM TSMTEST : q dbv Volume Name CopyVolume Name CopyVolume Name Copy (Copy 1) Status (Copy 2) Status (Copy 3) Status -- -- -- /opt/tivoli/tsm- Sync'dUndef- Undef- /server/bin/db- ined ined .dsm /tsmpool/db/db2- Sync'dUndef- Undef- .dsmined ined /tsmpool/db/db1- Sync'dUndef- Undef- .dsmined ined /home/tsmdb/db3- Sync'dUndef- Undef- .dsmined ined /tsmpool/db/db4- Sync'dUndef- Undef- .dsmined ined /tsmpool/db/db5- Sync'dUndef- Undef- .dsmined ined Clark, Robert A [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/02/2008 01:17 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? I would try doing a full DB backup. Can you send q dbv output too? [RC] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 9:20 AM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Where is the missing 38GB? I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals
Re: Where is the missing 38GB?
This is a test server, previously decommission from use about 9-months ago. I unloaded/reloaded the previous production DB to a new server. Then I went through and deleted everything. Don't keep us in suspense...and the answer is ? Mark Stapleton [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/02/2008 01:21 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? If you have a habit of deleting filespaces on this test box, that would explain it. You'd be seeing the fragmentation that happens to the TSM db when such brute-force deletions are done. I had several customers that had similar issues with their production boxes. -- Mark Stapleton ([EMAIL PROTECTED]) CDW Berbee System engineer 7145 Boone Avenue North, Suite 140 Brooklyn Park MN 55428-1511 763-592-5963 www.berbee.com -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 12:02 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? Or better yet, I could just reformat everything, since this is a test system. I just want to know why.. Yeah, yeah, I realize V6.1 with DB2 will fix all DB issues.; Schneider, Jim [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/02/2008 12:58 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? I'm waiting for the firestorm of comments following this suggestions. (Sitting back, waiting for fireworks.) -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Gee, Norman Sent: Tuesday, December 02, 2008 11:52 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? The dreaded DSMSERV DUMPDB and DSMSERV LOADDB (not recommended) DSMSERV LOADDB (Reload the database) Use this command to reload a Tivoli Storage Manager database in optimal order. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 9:44 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: Where is the missing 38GB? 12/2/2008 12:41:46 PM ANR0984I Process 1 for ESTIMATE DBREORG started in the BACKGROUND at 12:41:46 PM. 12/2/2008 12:41:46 PM ANR1782W ESTIMATE DBREORG process 1 started - server performance may be degraded while this process is running. 12/2/2008 12:41:46 PM ANR0405I Session 78 ended for administrator ZFORRAY (WinNT). 12/2/2008 12:41:53 PM ANR1784I A database reorganization would reduce the database utilization by an estimated 0 MB. 12/2/2008 12:41:53 PM ANR0987I Process 1 for ESTIMATE DBREORG running in the BACKGROUND processed 1344 items with a completion state of SUCCESS at 12:41:54 PM. 12/2/2008 12:41:53 PM ANR0381I Buffer pool statistics were successfully reset. On 02/12, Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: ... I did a DSMSERV AUDITDB FIX=YES and the only thing it complained (and fixed) about was old schedules for non-existing nodes. Also did an EXPIRE INVENTORY. The DB is probably fragmented. Try: ESTIMATE DBREORGSTATS Q DBVOL F=D
Re: Where is the missing 38GB?
That is kind of interesting, of the 15,694 used DB pages, you backed up 68,561. Here we have only been able to backup the used pages once per run. Andy Huebner -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 1:04 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? No change... 12/2/2008 1:53:43 PM ANR2017I Administrator ZFORRAY issued command: BACKUP DB dev=FILEDEVCLASS typ=Full scr=Yes 12/2/2008 1:54:49 PM ANR0984I Process 2 for DATABASE BACKUP started in the BACKGROUND at 01:54:50 PM. 12/2/2008 1:54:49 PM ANR2280I Full database backup started as process 2. 12/2/2008 1:54:49 PM ANR8340I FILE volume /tsmpool/filedirs/28244090.DBB mounted. 12/2/2008 1:54:49 PM ANR0513I Process 2 opened output volume /tsmpool/filedirs/28244090.DBB. 12/2/2008 1:54:49 PM ANR1360I Output volume /tsmpool/filedirs/28244090.DBB opened (sequence number 1). 12/2/2008 1:56:41 PM ANR4554I Backed up 34048 of 68561 database pages. 12/2/2008 1:57:11 PM ANR4554I Backed up 57024 of 68561 database pages. 12/2/2008 1:57:41 PM ANR4554I Backed up 61952 of 68561 database pages. 12/2/2008 1:58:11 PM ANR4554I Backed up 67392 of 68561 database pages. 12/2/2008 1:58:17 PM ANR1361I Output volume /tsmpool/filedirs/28244090.DBB closed. 12/2/2008 1:58:17 PM ANR0515I Process 2 closed volume /tsmpool/filedirs/28244090.DBB. 12/2/2008 1:58:17 PM ANR4550I Full database backup (process 2) complete, 68561 pages copied. 12/2/2008 1:58:17 PM ANR0985I Process 2 for DATABASE BACKUP running in the BACKGROUND completed with completion state SUCCESS at 01:58:18 PM. 1:59:53 PM TSMTEST : q db f=d Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,694 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 34,826 Cache Hit Pct.: 61.80 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 0 Changed Since Last Backup (MB): 0.04 Percentage Changed: 0.07 Last Complete Backup Date/Time: 12/02/2008 13:54:50 Estimate of Recoverable Space (MB): 0 Last Estimate of Recoverable Space (MB): 12/02/2008 12:41:54 --- 2:01:02 PM TSMTEST : q dbv Volume Name CopyVolume Name CopyVolume Name Copy (Copy 1) Status (Copy 2) Status (Copy 3) Status -- -- -- /opt/tivoli/tsm- Sync'dUndef- Undef- /server/bin/db- ined ined .dsm /tsmpool/db/db2- Sync'dUndef- Undef- .dsmined ined /tsmpool/db/db1- Sync'dUndef- Undef- .dsmined ined /home/tsmdb/db3- Sync'dUndef- Undef- .dsmined ined /tsmpool/db/db4- Sync'dUndef- Undef- .dsmined ined /tsmpool/db/db5- Sync'dUndef- Undef- .dsmined ined Clark, Robert A [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/02/2008 01:17 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? I would try doing a full DB backup. Can you send q dbv output too? [RC] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 9:20 AM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Where is the missing 38GB? I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct
Re: Where is the missing 38GB?
I caught that and also thought it was odd Something is definitly wrong in the DB pointers. I would have thought the auditdb would have fixed it. Maybe there are extra auditdb keywords I need to use to perform a more thorough scrubbing? Huebner,Andy,FORT WORTH,IT [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/02/2008 02:51 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? That is kind of interesting, of the 15,694 used DB pages, you backed up 68,561. Here we have only been able to backup the used pages once per run. Andy Huebner -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 1:04 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? No change... 12/2/2008 1:53:43 PM ANR2017I Administrator ZFORRAY issued command: BACKUP DB dev=FILEDEVCLASS typ=Full scr=Yes 12/2/2008 1:54:49 PM ANR0984I Process 2 for DATABASE BACKUP started in the BACKGROUND at 01:54:50 PM. 12/2/2008 1:54:49 PM ANR2280I Full database backup started as process 2. 12/2/2008 1:54:49 PM ANR8340I FILE volume /tsmpool/filedirs/28244090.DBB mounted. 12/2/2008 1:54:49 PM ANR0513I Process 2 opened output volume /tsmpool/filedirs/28244090.DBB. 12/2/2008 1:54:49 PM ANR1360I Output volume /tsmpool/filedirs/28244090.DBB opened (sequence number 1). 12/2/2008 1:56:41 PM ANR4554I Backed up 34048 of 68561 database pages. 12/2/2008 1:57:11 PM ANR4554I Backed up 57024 of 68561 database pages. 12/2/2008 1:57:41 PM ANR4554I Backed up 61952 of 68561 database pages. 12/2/2008 1:58:11 PM ANR4554I Backed up 67392 of 68561 database pages. 12/2/2008 1:58:17 PM ANR1361I Output volume /tsmpool/filedirs/28244090.DBB closed. 12/2/2008 1:58:17 PM ANR0515I Process 2 closed volume /tsmpool/filedirs/28244090.DBB. 12/2/2008 1:58:17 PM ANR4550I Full database backup (process 2) complete, 68561 pages copied. 12/2/2008 1:58:17 PM ANR0985I Process 2 for DATABASE BACKUP running in the BACKGROUND completed with completion state SUCCESS at 01:58:18 PM. 1:59:53 PM TSMTEST : q db f=d Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,694 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 34,826 Cache Hit Pct.: 61.80 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 0 Changed Since Last Backup (MB): 0.04 Percentage Changed: 0.07 Last Complete Backup Date/Time: 12/02/2008 13:54:50 Estimate of Recoverable Space (MB): 0 Last Estimate of Recoverable Space (MB): 12/02/2008 12:41:54 --- 2:01:02 PM TSMTEST : q dbv Volume Name CopyVolume Name CopyVolume Name Copy (Copy 1) Status (Copy 2) Status (Copy 3) Status -- -- -- /opt/tivoli/tsm- Sync'dUndef- Undef- /server/bin/db- ined ined .dsm /tsmpool/db/db2- Sync'dUndef- Undef- .dsmined ined /tsmpool/db/db1- Sync'dUndef- Undef- .dsmined ined /home/tsmdb/db3- Sync'dUndef- Undef- .dsmined ined /tsmpool/db/db4- Sync'dUndef- Undef- .dsmined ined /tsmpool/db/db5- Sync'dUndef- Undef- .dsmined ined Clark, Robert A [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/02/2008 01:17 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? I would try doing a full DB backup. Can you send q dbv output too? [RC] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU Sent: Tuesday, December 02, 2008 9:20 AM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Where is the missing 38GB? I have a test TSM server (5.5.1) which is
Re: Where is the missing 38GB?
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU This is a test server, previously decommission from use about 9-months ago. I unloaded/reloaded the previous production DB to a new server. Then I went through and deleted everything. Don't keep us in suspense...and the answer is ? The answer is DSMSERV UNLOADDB followed by DSMSERV LOADDB. And even then, any sort of major fragmentation will soon return. This is why the best answer to moving portions of a TSM server to another box (for load sharing, for instance) is to start with a fresh box and do IMPORT NODE work. -- Mark Stapleton ([EMAIL PROTECTED]) CDW Berbee System engineer 7145 Boone Avenue North, Suite 140 Brooklyn Park MN 55428-1511 763-592-5963 www.berbee.com
Re: Where is the missing 38GB?
Been there...done that..doing it right now...up to 5-TSM servers and possibly another one in the futureconstantly running EX/IMPORTS moving nodes around to balance the load. Mark Stapleton [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/02/2008 03:13 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Zoltan Forray/AC/VCU This is a test server, previously decommission from use about 9-months ago. I unloaded/reloaded the previous production DB to a new server. Then I went through and deleted everything. Don't keep us in suspense...and the answer is ? The answer is DSMSERV UNLOADDB followed by DSMSERV LOADDB. And even then, any sort of major fragmentation will soon return. This is why the best answer to moving portions of a TSM server to another box (for load sharing, for instance) is to start with a fresh box and do IMPORT NODE work. -- Mark Stapleton ([EMAIL PROTECTED]) CDW Berbee System engineer 7145 Boone Avenue North, Suite 140 Brooklyn Park MN 55428-1511 763-592-5963 www.berbee.com
Re: Where is the missing 38GB?
Too bad the database seems to be defective; otherwise you could use the DELete DBVolume technique (see IBM Technote 1208572) to progressively shift the db data around, with an added volume to get things started. Richard Sims
Re: Where is the missing 38GB?
I think I have tried that, before. I will play around some. I thought for sure you would have some secret, hidden, magical auditdb parm to fix this, Richard ;--)) Richard Sims [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 12/02/2008 03:54 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Where is the missing 38GB? Too bad the database seems to be defective; otherwise you could use the DELete DBVolume technique (see IBM Technote 1208572) to progressively shift the db data around, with an added volume to get things started. Richard Sims
Re: Where is the missing 38GB?
On Dec 2, 2008, at 3:55 PM, Zoltan Forray/AC/VCU wrote: I think I have tried that, before. I will play around some. I thought for sure you would have some secret, hidden, magical auditdb parm to fix this, Richard ;--)) Recall in the movie Labyrinth, there was a Bog of Eternal Stench. One does not venture in there willingly. R.
Re: Where is the missing 38GB?
Ok, this is an almost empty database with a lot of fragmentation. I'd say, from what I read in the thread, that an unload/load at this point will remove a lot of fragmentation, even though the estimate says nill. The other option is to run an export server, reformat everything, and then import server. You'll probably want to save your devconf.txt so you can easily recreate the stgpools, devclasses and server2server comms you might have. On Dec 2, 2008, at 18:20 , Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: 12:15:11 PM TSMTEST : q stg Storage Device EstimatedPctPct High Low Next Stora- Pool NameClass NameCapacity Util Migr Mig Mig ge Pool Pct Pct --- -- -- - - --- --- ARCHIVEPOOL DISK 0.0 M0.00.090 30 BACKUPPOOL DISK 123 G0.00.090 30 COPYPOOL-I- IBM3583-10.0 M0.0 BM3583-1 COPYPOOL-I- IBM3583-20.0 M0.0 BM3583-2 IBM3494-35- 3592E05 0.0 M0.00.090 70 92 I did a DSMSERV AUDITDB FIX=YES and the only thing it complained (and fixed) about was old schedules for non-existing nodes. Also did an EXPIRE INVENTORY. -- Met vriendelijke groeten, Remco Post [EMAIL PROTECTED] +31 6 248 21 622
Re: Where is the missing 38GB?
How about a q db f=d now that the estimate dbreorgstats is finished? The q dbv f=d doesn't really give anything that's affected by the estimate. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Remco Post Sent: Tuesday, December 02, 2008 3:04 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Where is the missing 38GB? Ok, this is an almost empty database with a lot of fragmentation. I'd say, from what I read in the thread, that an unload/load at this point will remove a lot of fragmentation, even though the estimate says nill. The other option is to run an export server, reformat everything, and then import server. You'll probably want to save your devconf.txt so you can easily recreate the stgpools, devclasses and server2server comms you might have. On Dec 2, 2008, at 18:20 , Zoltan Forray/AC/VCU wrote: I have a test TSM server (5.5.1) which is producing some strange DB statistics. ** *** --- Q DB F=D ** Available Space (MB): 56,336 Assigned Capacity (MB): 53,264 Maximum Extension (MB): 3,072 Maximum Reduction (MB): 14,360 Page Size (bytes): 4,096 Total Usable Pages: 13,635,584 Used Pages: 15,676 Pct Util: 0.1 Max. Pct Util: 0.1 Physical Volumes: 6 Buffer Pool Pages: 131,072 Total Buffer Requests: 249 Cache Hit Pct.: 100.00 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 4 Changed Since Last Backup (MB): 211.15 Percentage Changed: 344.82 Last Complete Backup Date/Time: 08/20/2008 09:49:38 Estimate of Recoverable Space (MB): Last Estimate of Recoverable Space (MB): With an assigned capacity of 52GB yet only 0.1% utilized (52MB?), it says I can only reduce the DB by 13GB So, where is the remaining 38GB of DB usage ? There are 5-Disk STG volumes (empty/0% utilized), 2-Nodes with NO filespaces, defined. Q STG shows: 12:15:11 PM TSMTEST : q stg Storage Device EstimatedPctPct High Low Next Stora- Pool NameClass NameCapacity Util Migr Mig Mig ge Pool Pct Pct --- -- -- - - --- --- ARCHIVEPOOL DISK 0.0 M0.00.090 30 BACKUPPOOL DISK 123 G0.00.090 30 COPYPOOL-I- IBM3583-10.0 M0.0 BM3583-1 COPYPOOL-I- IBM3583-20.0 M0.0 BM3583-2 IBM3494-35- 3592E05 0.0 M0.00.090 70 92 I did a DSMSERV AUDITDB FIX=YES and the only thing it complained (and fixed) about was old schedules for non-existing nodes. Also did an EXPIRE INVENTORY. -- Met vriendelijke groeten, Remco Post [EMAIL PROTECTED] +31 6 248 21 622
