Re: [Veritas-bu] Expired Backup ID Doesn't Free Up Some Space
I am by no means an expert in TSM. I did manage mainframe HSM environments for many years (and still do, although it doesn't need much tending to). And I dabbled with TSM for a few months. But from what I know, yes, I think you are right about reclaim (it is called recycle on the mainframe). It moves unexpired data from one tape to another, thereby freeing up the space on the source tape that was holding expired images. Cheers Dean On Wed, Sep 29, 2010 at 10:44 PM, Mark Hickey mark.hic...@hds.com wrote: I knew someone would pull out some niche backup product from some tiny vendor to hit me in the head with….:-) This is the last I thing I will add to the discussion, as it is a bit off-topic, but for my own curiosity… Dean, are you talking about reclaim? Because I don’t think that is exactly the same thing, though some might argue it is just semantics. If I understand correctly, reclaim copies unexpired images off of partially full tapes onto partially full tapes to increase both the number of full tapes and the number of empty tapes based on a threshold of unexpired capacity. If this is what you are talking about, then while it may in the end support some of what Adrian is doing, it is not what I was referring to. I was saying that I don’t think any backup applications notice an expired image in the middle of a tape, mark the tape used by that image as available, and then write new images on that same space on that same tape. Please help me understand TSM’s capabilities better if I am misinformed about this. Mark Dean wrote: IBM does tape recycling, and has for decades, first on the mainframe, and TSM is heavily based on the same concepts as HSM on the mainframe. They even called it recycle F DFHSM,RECYCLE ALL P(30) EX It means you can mix up your retentions on a single piece of tape media, but it also means you have to dedicate a good proportion of your tape hardware/time to looking after the recycle process (ie - basically doing tape-to-tape copies every day to filter out the expired data). There are pros and cons. Sorry, I have nothing to add to what Mark has said, other than to call out his no vendor does it :) Mark Mark Hickey Principal Technical Consultant HITACHI DATA SYSTEMS 62 Beach Road North Weymouth, MA 02191 O 781.331.3508 C 781.254.6441 mark.hic...@hds.com ___ Veritas-bu maillist - veritas...@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Expired Backup ID Doesn't Free Up Some Space
IBM does tape recycling, and has for decades, first on the mainframe, and TSM is heavily based on the same concepts as HSM on the mainframe. They even called it recycle F DFHSM,RECYCLE ALL P(30) EX It means you can mix up your retentions on a single piece of tape media, but it also means you have to dedicate a good proportion of your tape hardware/time to looking after the recycle process (ie - basically doing tape-to-tape copies every day to filter out the expired data). There are pros and cons. Sorry, I have nothing to add to what Mark has said, other than to call out his no vendor does it :) Cheers, Dean On Wed, Sep 29, 2010 at 3:40 AM, Mark Hickey mark.hic...@hds.com wrote: Adrian, Your problem , especially in the =case of physical tape, is that you put multiple retentions on the sane tape. I understand that in the short term this might reduce an initial tape purchase, but in the long term, you probably end up buying more tapes because the ones you are using contain 50% (or more) expired images, and at least one image that has not expired. Since it is very complex to try to “recycle” part of a tape cartridge, I don’t think any vendor does it. You have two basic options here: 1. turn of “Allow multiple retentions per tape” an create separate tape pools for policies with different retentions to pull from 2. Suspend the tapes from each backup session until all the images have expired. Mark Mark Hickey Principal Technical Consultant HITACHI DATA SYSTEMS 62 Beach Road North Weymouth, MA 02191 O 781.331.3508 C 781.254.6441 mark.hic...@hds.com ___ Veritas-bu maillist - veritas...@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Expired Backup ID Doesn't Free Up Some Space
Hi Judy, Thank you for the article, it's very useful clear to me. Best Regards, Adrian -Original Message- From: judy_hinchcli...@administaff.com [mailto:judy_hinchcli...@administaff.com] Sent: 28 September 2010 21:23 To: Adrian Soetanto; veritas-bu@mailman.eng.auburn.edu Subject: RE: Expired Backup ID Doesn't Free Up Some Space https://www-secure.symantec.com/connect/blogs/understanding-how-netbacku p-writes-tape I did a small write up on how NB writes to tapes and how expiring images on a tape works. This may give you a better understanding. -Original Message- From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of Adrian Soetanto Sent: Monday, September 27, 2010 4:37 AM To: veritas-bu@mailman.eng.auburn.edu Subject: [Veritas-bu] Expired Backup ID Doesn't Free Up Some Space Dear all, I have a problem with expired backup images that are not deleted as soon as it's expired, expecially the ones that are backed up using cartridge volume pool (not a staging disk). Here is the condition: Suppose I have a volume pool named 'BMR' and all cartridges/media IDs in it are full with multiple retention levels. Some backup IDs that were using 'BMR' volume pool were expired a days ago, but all media IDs in 'BMR' volume pool are still having the same media status (full with multiple retention levels). I can force the media ID (not the backup ID) to be expired and free it up by using this command: /usr/openv/netbackup/bin/admincmd/bpexpdate -m media_id -d 0 But, by using the above command, it will delete all of the backup IDs in that media ID, which I don't want it. What I want is to delete only certain backup IDs in a media ID and free up some space in that media ID. I've tried to use this command: /usr/openv/netbackup/bin/admincmd/bpexpdate -backupid backup_id -d 0 But it doesn't free up some space in the media ID, it just expires the backup ID. I've waited for the 'Image Cleanup' to run and hope it will free up some space at the above media ID. The Image Cleanup was finished successfully, but it didn't free up some space at the above media ID. From the Image Cleanup log, I guess it just free up some space at the staging disk only, doesn't it? If it does, then how can I force the expired backup ID to be able to free up some space in its media ID as soon as it was expired? Thank you for helping. -- Best Regards, Adrian ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Expired Backup ID Doesn't Free Up Some Space
Adrian I dont think this is the behaviour. From my view, until all images are deleted or expired, the tape is then free for Backups. Simon -Original Message- From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of Adrian Soetanto Sent: Monday, September 27, 2010 10:37 AM To: veritas-bu@mailman.eng.auburn.edu Subject: [Veritas-bu] Expired Backup ID Doesn't Free Up Some Space Dear all, I have a problem with expired backup images that are not deleted as soon as it's expired, expecially the ones that are backed up using cartridge volume pool (not a staging disk). Here is the condition: Suppose I have a volume pool named 'BMR' and all cartridges/media IDs in it are full with multiple retention levels. Some backup IDs that were using 'BMR' volume pool were expired a days ago, but all media IDs in 'BMR' volume pool are still having the same media status (full with multiple retention levels). I can force the media ID (not the backup ID) to be expired and free it up by using this command: /usr/openv/netbackup/bin/admincmd/bpexpdate -m media_id -d 0 But, by using the above command, it will delete all of the backup IDs in that media ID, which I don't want it. What I want is to delete only certain backup IDs in a media ID and free up some space in that media ID. I've tried to use this command: /usr/openv/netbackup/bin/admincmd/bpexpdate -backupid backup_id -d 0 But it doesn't free up some space in the media ID, it just expires the backup ID. I've waited for the 'Image Cleanup' to run and hope it will free up some space at the above media ID. The Image Cleanup was finished successfully, but it didn't free up some space at the above media ID. From the Image Cleanup log, I guess it just free up some space at the staging disk only, doesn't it? If it does, then how can I force the expired backup ID to be able to free up some space in its media ID as soon as it was expired? Thank you for helping. -- Best Regards, Adrian ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu This email (including any attachments) may contain confidential and/or privileged information or information otherwise protected from disclosure. If you are not the intended recipient, please notify the sender immediately, do not copy this message or any attachments and do not use it for any purpose or disclose its content to any person, but delete this message and any attachments from your system. Astrium disclaims any and all liability if this email transmission was virus corrupted, altered or falsified. -o- Astrium Limited, Registered in England and Wales No. 2449259 Registered Office: Gunnels Wood Road, Stevenage, Hertfordshire, SG1 2AS, England ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Expired Backup ID Doesn't Free Up Some Space
You cannot expire individual images on a media and re-use that space. Once a certain image has been written, NetBackup will not overwrite that position on the media until the entire tape has expired. -Jonathan -Original Message- From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of Adrian Soetanto Sent: Monday, September 27, 2010 5:37 AM To: veritas-bu@mailman.eng.auburn.edu Subject: [Veritas-bu] Expired Backup ID Doesn't Free Up Some Space Dear all, I have a problem with expired backup images that are not deleted as soon as it's expired, expecially the ones that are backed up using cartridge volume pool (not a staging disk). Here is the condition: Suppose I have a volume pool named 'BMR' and all cartridges/media IDs in it are full with multiple retention levels. Some backup IDs that were using 'BMR' volume pool were expired a days ago, but all media IDs in 'BMR' volume pool are still having the same media status (full with multiple retention levels). I can force the media ID (not the backup ID) to be expired and free it up by using this command: /usr/openv/netbackup/bin/admincmd/bpexpdate -m media_id -d 0 But, by using the above command, it will delete all of the backup IDs in that media ID, which I don't want it. What I want is to delete only certain backup IDs in a media ID and free up some space in that media ID. I've tried to use this command: /usr/openv/netbackup/bin/admincmd/bpexpdate -backupid backup_id -d 0 But it doesn't free up some space in the media ID, it just expires the backup ID. I've waited for the 'Image Cleanup' to run and hope it will free up some space at the above media ID. The Image Cleanup was finished successfully, but it didn't free up some space at the above media ID. From the Image Cleanup log, I guess it just free up some space at the staging disk only, doesn't it? If it does, then how can I force the expired backup ID to be able to free up some space in its media ID as soon as it was expired? Thank you for helping. -- Best Regards, Adrian ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Expired Backup ID Doesn't Free Up Some Space
On Mon, Sep 27, 2010 at 4:36 AM, Adrian Soetanto adrian.soeta...@bentoel.co.id wrote: What I want is to delete only certain backup IDs in a media ID and free up some space in that media ID. I've tried to use this command: /usr/openv/netbackup/bin/admincmd/bpexpdate -backupid backup_id -d 0 But it doesn't free up some space in the media ID, it just expires the backup ID. This will work for random access devices like disk but it won't work for sequential devices like tape. Tapes in NetBackup are written sequentially and do not allow pieces in the middle to be deleted. Even if you delete the last image on the tape, I'm not sure that NetBackup will truncate the tape and put the end-of-tape marker at the end of the last image. This is one of the reasons why it's a bad idea to mix retentions on the same tape. .../Ed Ed Wilts, RHCE, BCFP, BCSD, SCSP, SCSE ewi...@ewilts.org Linkedin http://www.linkedin.com/in/ewilts ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Expired Backup ID Doesn't Free Up Some Space
On Mon, Sep 27, 2010 at 08:28:07AM -0400, Martin, Jonathan wrote: You cannot expire individual images on a media and re-use that space. Once a certain image has been written, NetBackup will not overwrite that position on the media until the entire tape has expired. Right. And it's not just NBU. It's a limitation on (modern) tape media. If you write to a tape device, all the information after that point becomes inaccessible. So backup programs can't fill in holes in the tape. If you want to free up space like this you need to duplicate any images that remain on the tape to a new tape, then recycle the media entirely. -- Darren ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
