Re: Monthly Backups, ...again!
Several reasons:
1. I already have the exact file versions I want to have in TSM, and have
no functional reason to create another set of tapes. It is offensive to
the spirit of TSM to create an additional file copy, especially when I
have no capability to use them to locally restore the node.
2. Tapes (and tape drive usage) are expensive. I get one tape per node;
when you have 100GB tapes and 20GB nodes, this is a problem.
3. There is no second copy. If the data are that important, don't I want
media protection, especially if I'm keeping the data 7 years? And how do
I recopy the data periodically, as required for long term data retention?
And how do I migrate to new media?
4. In noncollocated environments, I will have to mount a number of tapes
to create the backup set at the end of the month, and I will have to do it
over and over for each node.
5. Backupsets are very inconvenient to restore from. You can either
restore the entire dataset, or use the commandline to try to pick out
exactly the file you are looking for. If you make a mistake, it searches
the whole thing before it fails, and you start over.
6. The backupset tapes are another thing to manage. At one time they did
not expire properly; I don't know if that has been fixed or not.
Sorry about the rantish tone, but you did ask...
_
William Mansfield
Senior Consultant
Solution Technology, Inc
Burak Demircan <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
04/16/2002 07:58 AM
Please respond to "ADSM: Dist Stor Manager"
To: [EMAIL PROTECTED]
cc:
Subject:Re: Monthly Backups, ...again!
Hi,
Why dont you think of backupsets? They do not require database usage and
easy
to use?
Just create backupset at the end each of period (year or month) and send
them
to
offsite with the volume history file. What else could be missing in that
senario? I think : nothing.
Regards,
Burak
[EMAIL PROTECTED]
Sent by: [EMAIL PROTECTED]
16.04.2002 15:53
Please respond to ADSM-L
To: [EMAIL PROTECTED]
cc:
Subject: Re: Monthly Backups, ...again!
I only wish I had a way to assign different retentions to active files, I
don't actually have one.
While I'm wishing...I wish I could pick an historical point in time, and
reset the retention for the versions corresponding to that point in time
to an arbitrary value.
_
William Mansfield
Senior Consultant
Solution Technology, Inc
"Don France (TSMnews)" <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
04/13/2002 03:58 PM
Please respond to "ADSM: Dist Stor Manager"
To: [EMAIL PROTECTED]
cc:
Subject: Re: Monthly Backups, ...again!
Bill,
Your arguments are excellent, and I have one large client dealing with the
health
records issue --- they need to save patient records for 32 years; we are
liking
the export solution, once a year, for those nodes (in order to control TSM
db
size) -- and, continue storing their data in archive storage so we have
access to
all versions generated, then (AFTER THE FACT) deleting extra versions from
each month --- save 2 or 3, delete the rest from archive storage, using a
"smart"
script in concert with the dba's technique of naming the files being
stored.
However I only know ONE (simple) way to accomplish the desired, stated
results ---
month-end snapshot kept for X months
or years; ie, a backupset of the desired file systems on specific nodes.
The limitation
of this solution is it only captures files in backup storage; you need a
different
answer for database backups stored in archive storage -- my DBA's love
using
archive storage, and I have no argument against it, as they only care
about
time
(have no need or interest in counting versions, etc), and they must save
daily
backups for 4-14 days, weekly's for 6 weeks, monthly's for 6-15 months,
etc.
If you have a method that allows you "mark" the currently active versions
of
backup
files with a different retention than others, I think it would be new news
to most of us...
please share.
Thanks,
Don
Don France
Technical Architect - Tivoli Certified Consultant
Professional Association of Contract Employees (P.A.C.E.)
San Jose, CA
(408) 257-3037
[EMAIL PROTECTED]
- Original Message -
From: "Bill Mansfield" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Thursday, April 04, 2002 9:55 AM
Subject: Re: Monthly Backups, ...again!
> There is a good reason it keeps coming up: legitimate business
> requirements.
>
> The suits (auditors, IRS, corp counsel, HIPAA, etc) demand
Re: Monthly Backups, ...again!
Hi, Why dont you think of backupsets? They do not require database usage and easy to use? Just create backupset at the end each of period (year or month) and send them to offsite with the volume history file. What else could be missing in that senario? I think : nothing. Regards, Burak [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 16.04.2002 15:53 Please respond to ADSM-L To: [EMAIL PROTECTED] cc: Subject: Re: Monthly Backups, ...again! I only wish I had a way to assign different retentions to active files, I don't actually have one. While I'm wishing...I wish I could pick an historical point in time, and reset the retention for the versions corresponding to that point in time to an arbitrary value. _ William Mansfield Senior Consultant Solution Technology, Inc "Don France (TSMnews)" <[EMAIL PROTECTED]> Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> 04/13/2002 03:58 PM Please respond to "ADSM: Dist Stor Manager" To: [EMAIL PROTECTED] cc: Subject: Re: Monthly Backups, ...again! Bill, Your arguments are excellent, and I have one large client dealing with the health records issue --- they need to save patient records for 32 years; we are liking the export solution, once a year, for those nodes (in order to control TSM db size) -- and, continue storing their data in archive storage so we have access to all versions generated, then (AFTER THE FACT) deleting extra versions from each month --- save 2 or 3, delete the rest from archive storage, using a "smart" script in concert with the dba's technique of naming the files being stored. However I only know ONE (simple) way to accomplish the desired, stated results --- month-end snapshot kept for X months or years; ie, a backupset of the desired file systems on specific nodes. The limitation of this solution is it only captures files in backup storage; you need a different answer for database backups stored in archive storage -- my DBA's love using archive storage, and I have no argument against it, as they only care about time (have no need or interest in counting versions, etc), and they must save daily backups for 4-14 days, weekly's for 6 weeks, monthly's for 6-15 months, etc. If you have a method that allows you "mark" the currently active versions of backup files with a different retention than others, I think it would be new news to most of us... please share. Thanks, Don Don France Technical Architect - Tivoli Certified Consultant Professional Association of Contract Employees (P.A.C.E.) San Jose, CA (408) 257-3037 [EMAIL PROTECTED] - Original Message - From: "Bill Mansfield" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, April 04, 2002 9:55 AM Subject: Re: Monthly Backups, ...again! > There is a good reason it keeps coming up: legitimate business > requirements. > > The suits (auditors, IRS, corp counsel, HIPAA, etc) demand to be able to > be able to reproduce any datum at given intervals for given durations. > Most often, that translates to restoring files that may change every day > to "month end" state for somewhere between 1 and 7 years. Sometime they > can identify the kinds of data they want, but it is expensive to > accurately identify the list of all files/directories required, so usually > you get a vague wave to "save everything". And of course, it's their > data, not yours, they have a right to keep as much as they want. Telling > them that TSM doesn't support their requirement just invites other > software vendors in the door since *they* handle this particular > requirement with ease (on paper). > > The number of days you can reasonably keep in an incremental backup > usually doesn't extend to forever. Archives sometimes don't cut it, > either in their traditional form or the instant form. You can't stand to > move that much data or use that many tapes - that's why you went > incremental forever in the first place. I really just to do some > operation that marks the current active version with a longer guaranteed > retention, without changing the retention of anything else. > > I don't want to restart the perennial discussion of truly long term > archival storage. It's reasonable to expect a backup system to maintain > internal compatibility for 7 years, and there are techniques for migrating > the data to newer media. > > Just my 5 cents worth (inflation). > _ > William Mansfield > Senior Consultant > Solution Technology,
Re: Monthly Backups, ...again!
I only wish I had a way to assign different retentions to active files, I
don't actually have one.
While I'm wishing...I wish I could pick an historical point in time, and
reset the retention for the versions corresponding to that point in time
to an arbitrary value.
_
William Mansfield
Senior Consultant
Solution Technology, Inc
"Don France (TSMnews)" <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
04/13/2002 03:58 PM
Please respond to "ADSM: Dist Stor Manager"
To: [EMAIL PROTECTED]
cc:
Subject:Re: Monthly Backups, ...again!
Bill,
Your arguments are excellent, and I have one large client dealing with the
health
records issue --- they need to save patient records for 32 years; we are
liking
the export solution, once a year, for those nodes (in order to control TSM
db
size) -- and, continue storing their data in archive storage so we have
access to
all versions generated, then (AFTER THE FACT) deleting extra versions from
each month --- save 2 or 3, delete the rest from archive storage, using a
"smart"
script in concert with the dba's technique of naming the files being
stored.
However I only know ONE (simple) way to accomplish the desired, stated
results ---
month-end snapshot kept for X months
or years; ie, a backupset of the desired file systems on specific nodes.
The limitation
of this solution is it only captures files in backup storage; you need a
different
answer for database backups stored in archive storage -- my DBA's love
using
archive storage, and I have no argument against it, as they only care
about
time
(have no need or interest in counting versions, etc), and they must save
daily
backups for 4-14 days, weekly's for 6 weeks, monthly's for 6-15 months,
etc.
If you have a method that allows you "mark" the currently active versions
of
backup
files with a different retention than others, I think it would be new news
to most of us...
please share.
Thanks,
Don
Don France
Technical Architect - Tivoli Certified Consultant
Professional Association of Contract Employees (P.A.C.E.)
San Jose, CA
(408) 257-3037
[EMAIL PROTECTED]
- Original Message -
From: "Bill Mansfield" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Thursday, April 04, 2002 9:55 AM
Subject: Re: Monthly Backups, ...again!
> There is a good reason it keeps coming up: legitimate business
> requirements.
>
> The suits (auditors, IRS, corp counsel, HIPAA, etc) demand to be able to
> be able to reproduce any datum at given intervals for given durations.
> Most often, that translates to restoring files that may change every day
> to "month end" state for somewhere between 1 and 7 years. Sometime they
> can identify the kinds of data they want, but it is expensive to
> accurately identify the list of all files/directories required, so
usually
> you get a vague wave to "save everything". And of course, it's their
> data, not yours, they have a right to keep as much as they want. Telling
> them that TSM doesn't support their requirement just invites other
> software vendors in the door since *they* handle this particular
> requirement with ease (on paper).
>
> The number of days you can reasonably keep in an incremental backup
> usually doesn't extend to forever. Archives sometimes don't cut it,
> either in their traditional form or the instant form. You can't stand
to
> move that much data or use that many tapes - that's why you went
> incremental forever in the first place. I really just to do some
> operation that marks the current active version with a longer guaranteed
> retention, without changing the retention of anything else.
>
> I don't want to restart the perennial discussion of truly long term
> archival storage. It's reasonable to expect a backup system to maintain
> internal compatibility for 7 years, and there are techniques for
migrating
> the data to newer media.
>
> Just my 5 cents worth (inflation).
> _
> William Mansfield
> Senior Consultant
> Solution Technology, Inc
>
>
>
>
>
> "Mr. Lindsay Morris" <[EMAIL PROTECTED]>
> Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
> 04/04/2002 10:04 AM
> Please respond to lmorris
>
>
> To: [EMAIL PROTECTED]
> cc:
> Subject:Re: Monthly Backups, ...again!
>
>
> This keeps coming up. It's the hardest thing about TSM, to sell users
on
> the way it works.
>
> Tivoli's Storage Vision whitepaper has a comparison of the benefits you
> get
> by NOT using this Grandfather-father-son technique, but I wish somebody
at
&
Re: Monthly Backups, ...again!
Bill, Your arguments are excellent, and I have one large client dealing with the health records issue --- they need to save patient records for 32 years; we are liking the export solution, once a year, for those nodes (in order to control TSM db size) -- and, continue storing their data in archive storage so we have access to all versions generated, then (AFTER THE FACT) deleting extra versions from each month --- save 2 or 3, delete the rest from archive storage, using a "smart" script in concert with the dba's technique of naming the files being stored. However I only know ONE (simple) way to accomplish the desired, stated results --- month-end snapshot kept for X months or years; ie, a backupset of the desired file systems on specific nodes. The limitation of this solution is it only captures files in backup storage; you need a different answer for database backups stored in archive storage -- my DBA's love using archive storage, and I have no argument against it, as they only care about time (have no need or interest in counting versions, etc), and they must save daily backups for 4-14 days, weekly's for 6 weeks, monthly's for 6-15 months, etc. If you have a method that allows you "mark" the currently active versions of backup files with a different retention than others, I think it would be new news to most of us... please share. Thanks, Don Don France Technical Architect - Tivoli Certified Consultant Professional Association of Contract Employees (P.A.C.E.) San Jose, CA (408) 257-3037 [EMAIL PROTECTED] - Original Message - From: "Bill Mansfield" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, April 04, 2002 9:55 AM Subject: Re: Monthly Backups, ...again! > There is a good reason it keeps coming up: legitimate business > requirements. > > The suits (auditors, IRS, corp counsel, HIPAA, etc) demand to be able to > be able to reproduce any datum at given intervals for given durations. > Most often, that translates to restoring files that may change every day > to "month end" state for somewhere between 1 and 7 years. Sometime they > can identify the kinds of data they want, but it is expensive to > accurately identify the list of all files/directories required, so usually > you get a vague wave to "save everything". And of course, it's their > data, not yours, they have a right to keep as much as they want. Telling > them that TSM doesn't support their requirement just invites other > software vendors in the door since *they* handle this particular > requirement with ease (on paper). > > The number of days you can reasonably keep in an incremental backup > usually doesn't extend to forever. Archives sometimes don't cut it, > either in their traditional form or the instant form. You can't stand to > move that much data or use that many tapes - that's why you went > incremental forever in the first place. I really just to do some > operation that marks the current active version with a longer guaranteed > retention, without changing the retention of anything else. > > I don't want to restart the perennial discussion of truly long term > archival storage. It's reasonable to expect a backup system to maintain > internal compatibility for 7 years, and there are techniques for migrating > the data to newer media. > > Just my 5 cents worth (inflation). > _ > William Mansfield > Senior Consultant > Solution Technology, Inc > > > > > > "Mr. Lindsay Morris" <[EMAIL PROTECTED]> > Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> > 04/04/2002 10:04 AM > Please respond to lmorris > > > To: [EMAIL PROTECTED] > cc: > Subject:Re: Monthly Backups, ...again! > > > This keeps coming up. It's the hardest thing about TSM, to sell users on > the way it works. > > Tivoli's Storage Vision whitepaper has a comparison of the benefits you > get > by NOT using this Grandfather-father-son technique, but I wish somebody at > Tivoli would come up with some better assistance to help us sell the > incremental-forever -ooops, progressive backup methodolgy - to non-techie > users. (Maybe it's there and I just don't know where to find it...?) > > I think Kelly Lipp has a good article on archiving and when it's sensible > - > maybe he'll post that link here again. > > Also, maybe some users have specific oddball scenarios they have run into > that require surprising policy settings. It would be interesting to hear > about those. Like, the user who goes on vacation for two weeks, and > manages > to trash here email file the day she leaves, doesn
Re: Monthly Backups, ...again!: The Real Issues
We are not calling them exactly Class 1, Class 2 and Class 3 but are using
the same rules for collocation. There is a slight modification - you can
"manually" collocate by creating Class 3a, Class 3b, etc. So the tape pool
for Class 3 would not be too big and on restore you will need
less tapes. And Class 3z may become Class 4 with
all-those-absolutely-non-critical-nodes and unlimited number of tapes in
it.
I also enjoy debates. And many threads on this list are about *proper* TSM
(now ISM) setup. I have seen a customer using Oracle database through ODBC
as plain dBase files with own relations logic on top of this. Guess what
was the performace. From this I cannot say Oracle is abnormally slow
database, right? Weird setup does not mean the product is bad.
That's why for example Incremental-forever does not work with TDP (now ISM
for ...) products. If MS Exchange had adopted this nice backup technique
and APIs allow some kind of mapping of IS to filespace name, Mailboxes to
directories and folders to files ... We could have incremental-forever
with restore granularity down to a folder and subfile backup can help us
to transfer only new/deleted messages. Or something even better. Same for
Oracle for example - instances may be same type of object as filespaces,
tablespaces - directories and tables vs files.
BTW: explaining to customer I prefer to use term "object" pointing that
for B/A client this is an ordinary file but for TDP is application
dependant. And from this point of view I explain why many good sides of
"Progressive Backup" go away.
Zlatko Krastev
IT Consultant
P.S. And completely new things demand from us to change our way of
thinking. Same as a century ago there was speed limit for cars of 5 km/h
and a man with red sign ought to walk in front of the car to warn the
public and prevent horses to panic!!! I am not kidding.
Zlatko
Please respond to "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
Sent by:"ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
cc:
Subject:Re: Monthly Backups, ...again!: The Real Issues
I enjoy TSM debates, but one that has not sold me yet is this particular
one. Yes, I come from a different environment (than Incremental-forever).
One of the biggest drawbacks of the old ADSM was the abnormal slow
restores
and this has been notified by a collegue of mine of why his previous job
did not get ADSM back in the early 90's. My suspicion was this
Incremental-forever and here I see why. It all relies on the collocate
issue. If you have numerous tapes and numerous drives than collocate will
make Incremental-forever a quck restore. I started with 2 drives 80
clients and 500 tapes leading to no-collocate. When restoring 2 GB+
restores that were spread out over 70-100 tapes this took almost 8 hours
(note this took about 1 year to get to 70-100 tapes). At our DR test we
noticed a diffence of 6 times a normal restore for no-collocate versus a
backupset. But on the other hand I brought that restore time down when
splitting the restore into 5 separate restores for 5 tape drives (I
currently have 6 drives). So the no-collocate will work if the right
amount of tape drives are available (our 3570 are pretty expensive). Now
in the real world you usually have 1-2 tape drives for restores
(dedicated)
and you cannot afford to send 80 tapes off every night for collocate, so
what we are gravitating to is a class1, class2, and class3 system. class1
will be our DRM critical restore with collocate onsite and offsite, class2
our noncritical but high restore class with collocate onsite and
noncollocate offsite. and then the good class3 with nocollocate onsite or
offsite. The monthlys or Absolutes will be used to stop the spreading of
info over 70 -100 tapes. It may not be true, but I seem to see the more
reclamation the more spreading of info. and I would have thought that the
reclamation would bring info closer together. All it takes is one time of
getting burned with the 70-100 tape restore spread. We have presently
moved data from one server to another 7 GB and used TSM to do this. If the
manual full had not be done the night before (which has happened) then I
still be restoring as we speak. There are more good things about fulls
than meets the eye. Now I know not all restores are 2 GB+ but I do see
more and more and we will all have to deal with it in our own ways. Just
1
way of dealing with it. Not the only but a way is better than no way.
"Seay, Paul"
cc:
Sent by: Subject: Re: Monthly Backups,
...again!: The Real Issues
"ADSM: Dist
Stor Manager"
<[EMAIL PROTECTED]
IST.EDU>
04/10/02 01:14
AM
Please respond
Re: Monthly Backups, ...again!: The Real Issues
Bill, Just out of curiosity, in your restore situation -- was it an NT platform (I see it must have been alot of tiny files)?!? If you are using NT, Netware or AIX as file server platforms, the DIRMC option pays for itself... BIG TIME. I recently responded to a customer, clearly a smaller shop, but the storage pool was non-collocated for over a year, using DLT library in a Win2K-to-Win2K context; there was a total RAID failure for the E: drive, but I originally engineered it with DIRMC on disk, migrated to FILE on disk, then copy pool'ed to tape. After about 30 hours, 1.6 million files and 316 GB of data were restored to the point-in-time specified as the last-known-good; this performance was achieved by using two concurrent restore sessions (across 10 high-level directories), CLASSIC restore (not the default), and the "dirsonly" first, then the data -- only slowdown was due to tape mounts (which were consolidated within each session), because the customer had more tapes than slots, so needed to respond to demand mount requests for tapes "mountable not in library". This experience is usually a wake-up call for the customer to evaluate RESTORE requirements; if 5-10 GB/Hr is satisfactory, then, so be it. 2GB restore in 8 hours, did you check your accounting records for media wait -- and does that reconcile with the time? How about the number of tape mounts (available from the summary records or count the approp. message # in the activity log)??? Sounds abit suspicious, to me. I've seen several ideas shared in this thread, any one of which could be the right answer for a given context; your 3-class system sounds interesting -- as does Bill Colwell's, and Paul's, and Nick's, and Alex. Also, with 5.1 the new IMAGE backup would seem a good substitute for monthly backupset. Ultimately, I like Jim Taylor's answer the best... get the dialog on RESTORE needs, then figure out what of the various suggestions will work for a given customer/server/class-of-servers. Of course, the key political question is truly to get a dialog on RESTORE REQUIREMENTS; focus on the business needs for that first, then benchmark and/or collaborate for possible solutions -- ultimately, the customer needs to take this seriously enough to "pay" for trial exercises a couple times a year... else, they're just putting their heads in the sand and courting disappointment. Regards, Don France Technical Architect - Tivoli Certified Consultant Professional Association of Contract Employees (P.A.C.E.) San Jose, CA (408) 257-3037 [EMAIL PROTECTED] - Original Message - From: "William Rosette" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Wednesday, April 10, 2002 6:12 AM Subject: Re: Monthly Backups, ...again!: The Real Issues > I enjoy TSM debates, but one that has not sold me yet is this particular > one. Yes, I come from a different environment (than Incremental-forever). > One of the biggest drawbacks of the old ADSM was the abnormal slow restores > and this has been notified by a collegue of mine of why his previous job > did not get ADSM back in the early 90's. My suspicion was this > Incremental-forever and here I see why. It all relies on the collocate > issue. If you have numerous tapes and numerous drives than collocate will > make Incremental-forever a quck restore. I started with 2 drives 80 > clients and 500 tapes leading to no-collocate. When restoring 2 GB+ > restores that were spread out over 70-100 tapes this took almost 8 hours > (note this took about 1 year to get to 70-100 tapes). At our DR test we > noticed a diffence of 6 times a normal restore for no-collocate versus a > backupset. But on the other hand I brought that restore time down when > splitting the restore into 5 separate restores for 5 tape drives (I > currently have 6 drives). So the no-collocate will work if the right > amount of tape drives are available (our 3570 are pretty expensive). Now > in the real world you usually have 1-2 tape drives for restores (dedicated) > and you cannot afford to send 80 tapes off every night for collocate, so > what we are gravitating to is a class1, class2, and class3 system. class1 > will be our DRM critical restore with collocate onsite and offsite, class2 > our noncritical but high restore class with collocate onsite and > noncollocate offsite. and then the good class3 with nocollocate onsite or > offsite. The monthlys or Absolutes will be used to stop the spreading of > info over 70 -100 tapes. It may not be true, but I seem to see the more > reclamation the more spreading of info. and I would have thought that the > reclamation would bring info closer together. All it takes is one time of > getting burned with the 70-100 tape restore spread. We have presently > moved data from one server to another 7 G
Re: Monthly Backups, ...again!: The Real Issues
I enjoy TSM debates, but one that has not sold me yet is this particular one. Yes, I come from a different environment (than Incremental-forever). One of the biggest drawbacks of the old ADSM was the abnormal slow restores and this has been notified by a collegue of mine of why his previous job did not get ADSM back in the early 90's. My suspicion was this Incremental-forever and here I see why. It all relies on the collocate issue. If you have numerous tapes and numerous drives than collocate will make Incremental-forever a quck restore. I started with 2 drives 80 clients and 500 tapes leading to no-collocate. When restoring 2 GB+ restores that were spread out over 70-100 tapes this took almost 8 hours (note this took about 1 year to get to 70-100 tapes). At our DR test we noticed a diffence of 6 times a normal restore for no-collocate versus a backupset. But on the other hand I brought that restore time down when splitting the restore into 5 separate restores for 5 tape drives (I currently have 6 drives). So the no-collocate will work if the right amount of tape drives are available (our 3570 are pretty expensive). Now in the real world you usually have 1-2 tape drives for restores (dedicated) and you cannot afford to send 80 tapes off every night for collocate, so what we are gravitating to is a class1, class2, and class3 system. class1 will be our DRM critical restore with collocate onsite and offsite, class2 our noncritical but high restore class with collocate onsite and noncollocate offsite. and then the good class3 with nocollocate onsite or offsite. The monthlys or Absolutes will be used to stop the spreading of info over 70 -100 tapes. It may not be true, but I seem to see the more reclamation the more spreading of info. and I would have thought that the reclamation would bring info closer together. All it takes is one time of getting burned with the 70-100 tape restore spread. We have presently moved data from one server to another 7 GB and used TSM to do this. If the manual full had not be done the night before (which has happened) then I still be restoring as we speak. There are more good things about fulls than meets the eye. Now I know not all restores are 2 GB+ but I do see more and more and we will all have to deal with it in our own ways. Just 1 way of dealing with it. Not the only but a way is better than no way. "Seay, Paul" cc: Sent by: Subject: Re: Monthly Backups, ...again!: The Real Issues "ADSM: Dist Stor Manager" <[EMAIL PROTECTED] IST.EDU> 04/10/02 01:14 AM Please respond to "ADSM: Dist Stor Manager" I would like to find the first customer out there with 20 clients (80+GB servers) that does not lose a full backup at least once a week and exceeds the backup window time and has no backup of that server. Or similarly on the incremental that is not restartable. TSM does checkpointing and completes a backup after restarting at the backup point. Heck, I have HALTed my TSM server in the middle of backups and watched them restart at were they left off. I did it to see what would happen. My Windows folks lost 1 backup that was in a unrecoverable initialization state at the time of the halt. And, that was easily restarted. There is no other viable product in the market that can do this. The argument is you cannot restore the incremental because it takes too long is bogus. I never could realize what I was missing in the conversation until I read Zlatko's explanation below. I forget that the Weekly FULL paradigm INCREMENTAL daily makes people think they have to restore each file to make up the incremental many times. When TSM just restores the right one to the point in time that you need recovery. There is an issue that concerns me that most folks forget about. Your DR offsite set may span a long period of time. After so long the tapes may not be readable because the DR site people dropped them a couple of times and you not know it. Two sets pose the same threat, but less so. So, if a tape is bad, you lose a lot of data, possibly a lot of servers if many are in the same storage pool and you do not have any collocation turned on. There are two solutions to this problem. If we had reclaim a tape if it was created more than x days ago, that would force reclamation on a tape no matter what. The other is a duplicate offsite pool. Neither of these capabilities are automatic today. To get the tapes to reclaim you have to do your own MOVE DATA RECONSTRUCT=YES for any tape last written to over x days (it would be nice to have an optional value to specify in the storage pool for this). The duplicate offsite requires you to run the BACKUP
Re: Monthly Backups, ...again!: The Real Issues
I would like to find the first customer out there with 20 clients (80+GB servers) that does not lose a full backup at least once a week and exceeds the backup window time and has no backup of that server. Or similarly on the incremental that is not restartable. TSM does checkpointing and completes a backup after restarting at the backup point. Heck, I have HALTed my TSM server in the middle of backups and watched them restart at were they left off. I did it to see what would happen. My Windows folks lost 1 backup that was in a unrecoverable initialization state at the time of the halt. And, that was easily restarted. There is no other viable product in the market that can do this. The argument is you cannot restore the incremental because it takes too long is bogus. I never could realize what I was missing in the conversation until I read Zlatko's explanation below. I forget that the Weekly FULL paradigm INCREMENTAL daily makes people think they have to restore each file to make up the incremental many times. When TSM just restores the right one to the point in time that you need recovery. There is an issue that concerns me that most folks forget about. Your DR offsite set may span a long period of time. After so long the tapes may not be readable because the DR site people dropped them a couple of times and you not know it. Two sets pose the same threat, but less so. So, if a tape is bad, you lose a lot of data, possibly a lot of servers if many are in the same storage pool and you do not have any collocation turned on. There are two solutions to this problem. If we had reclaim a tape if it was created more than x days ago, that would force reclamation on a tape no matter what. The other is a duplicate offsite pool. Neither of these capabilities are automatic today. To get the tapes to reclaim you have to do your own MOVE DATA RECONSTRUCT=YES for any tape last written to over x days (it would be nice to have an optional value to specify in the storage pool for this). The duplicate offsite requires you to run the BACKUP STG for each copy (it would be nice to specify multiple outs if that makes sense and is possible). The other thought that I have had is to have multiple copy pools and rotate through them updating each on on a cyclic basis. So, if you had 6 and you sent an update offsite weekly you would have 6 intact storage pools. But, the server overhead to do this is significant. Other FULL Backup tools have a way around this by accepting a week older backup of the data. Essentially, they have 6 copies of the data offsite. Just a matter of how much data they want to lose. The point here is plan your TSM environment to meet your business recovery needs and you will have a vastly superior solution than any other product can offer. -Original Message- From: Zlatko Krastev [mailto:[EMAIL PROTECTED]] Sent: Sunday, April 07, 2002 11:02 PM To: [EMAIL PROTECTED] Subject: Re: Monthly Backups, ...again! Similar to what I try to explain to customers: TSM does only incrementals but at the server they are transpatently merged. So you (I mean the customer) get restore from full backup. This is the best - quick backups and quick restores, no incrementals to apply, no full backups to transfer. Just after this start saying "Incremental-forever" or "Progressive Backups" mentioning the benefit. Zlatko Krastev IT Consultant Please respond to "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> Sent by:"ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] cc: Subject:Re: Monthly Backups, ...again! My current favorite way to explain Progressive Backups - "TSM updates the full backup incrementally." Get that one across, then explain versioning at the file level, and you've got them! Nick Cassimatis [EMAIL PROTECTED] Today is the tomorrow of yesterday.
Re: Monthly Backups, ...again!
Similar to what I try to explain to customers: TSM does only incrementals but at the server they are transpatently merged. So you (I mean the customer) get restore from full backup. This is the best - quick backups and quick restores, no incrementals to apply, no full backups to transfer. Just after this start saying "Incremental-forever" or "Progressive Backups" mentioning the benefit. Zlatko Krastev IT Consultant Please respond to "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> Sent by:"ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] cc: Subject:Re: Monthly Backups, ...again! My current favorite way to explain Progressive Backups - "TSM updates the full backup incrementally." Get that one across, then explain versioning at the file level, and you've got them! Nick Cassimatis [EMAIL PROTECTED] Today is the tomorrow of yesterday.
Re: Monthly Backups, ...again!
Thanks to all that replied! Looks like I am not alone!
I'll let you know the outcome...
Marc Levitan
Storage Manager
PFPC Global Fund Services
"Seay, Paul"
cc:
Sent by: Subject: Re: Monthly Backups, ...again!
"ADSM: Dist
Stor Manager"
<[EMAIL PROTECTED]
RIST.EDU>
04/04/2002
03:17 PM
Please
respond to
"ADSM: Dist
Stor Manager"
The question is what do you hope to restore from it. You will find that a
full backup like this applies to a specific application, not the entire
company. It is usually a small segment of the corporate data. In the
mainframe world with tapes for applications you just created a tape with a
specific retention, special backup of that information. There are probably
a thousand ways to accomplish this in the TSM open world. The most viable
way in the open systems world is to copy the data files to an archive area
that is archive/deleted and kept for the appropriate period of time. This
way the application has the responsibility to put the data in the location
at the appropriate time and the storage management solution now manages
that
area with a predefined set of criteria that is totally automated.
Yes, this requires a little bit of work for the application. But, if they
really want data recovery at an application level then this is the most
effective way. Everyone, knows the archive data areas then and where to
find the archived data.
-Original Message-
From: Marc Levitan [mailto:[EMAIL PROTECTED]]
Sent: Thursday, April 04, 2002 11:16 AM
To: [EMAIL PROTECTED]
Subject: Monthly Backups, ...again!
Has anyone had to defend themselves against the MONTHLY FULL BACKUP kept
for
a year scenario??? Business wants a monthly full backup to be kept for a
year. How have people dealt with this issue?
Thanks,
Marc Levitan
Storage Manager
PFPC Global Fund Services
- Forwarded by Marc D Levitan/PFPC/WES/PNC on 04/04/2002 11:15 AM -
Marc D
Levitan To: [EMAIL PROTECTED]
cc:
04/04/2002 Subject: Monthly Backups,
...again!
08:51 AM
A question was brought up while discussing retention policies.
Currently we have the following retentions:
PolicyPolicyMgmt Copy Versions Versions Retain Retain
DomainSet Name Class Group Data DataExtraOnly
NameName NameExists Deleted Versions Version
- - - - ---
COLD ACTIVECOLD STANDARD 215 30
NOVELLACTIVEDIRMC STANDARD301 120 365
NOVELLACTIVESTANDARD STANDARD301 120 365
RECON ACTIVEDIRMC STANDARD363 75 385
RECON ACTIVEMC_RECON STANDARD261 60 365
STANDARD ACTIVEDIRMC STANDARD261 60 365
STANDARD ACTIVESTANDARD STANDARD261 60 365
UNIX ACTIVEMC_UNIX STANDARD301 60 30
I believe that this provides for daily backups for over a month.
There was a request to have the following:
1) Daily backups for a week.
2) Weekly backups for a month.
3) Monthly backups for a year.
I believe we are providing 1 & 2. We are providing daily backups for a
month.
How can I provide monthly backups for a year?
I know that I could take monthly archives, but this would exceed our backup
windows and would increase our resources ( db, tapes, etc.) Also, I know we
could lengthen our retention policies. Also we could create backup sets.
(tons of tapes!)
How are other people handling this?
Thanks,
Marc Levitan
Storage Manager
PFPC Global Fund Services
Re: Monthly Backups, ...again!
Oh yes, I have had to defend myself along with many other *TSM'ers I
believe. I think Thomas Denier said it best "I would suggest that your
customer either give up on TSM or give up on preserving the current tape
handling practices."
I have forwarded this message and many others (there is plenty) from the
adsm.org archives to my management. That along with a few backupsets, the
pressure has stopped for now. :)
Date: Jan 16, 11:36
From: Thomas Denier <[EMAIL PROTECTED]>
>I am planning a TSM implementation ata customer who has another backup
> solution already in place. They want to slowly make TSM take over all
backups,
> but they don4t want to suddenly change the work done by the production
people,
> who take care of all tape movements. The present backup solution works
using
> different retention times for the backed up files: the daily backups are
kept
> for 20 days; the saturday ("weekly") files are kept for 40 days; the
monthly
> backup is kept for 100 days, and there is a "year" backup that is kept for
5
> years.
>
>The question is: I could make this using backups for the daily
processes,
> and to use archives for the other ones. But there are TDP4s involved
> (SQLServer, Exchange and DB2) who always do backups, and if I use
something
> like different management classes, I would get a lot of rebinding
problems.
>
>Would backupsets be the answer? Any ideas would be greatly appreciated.
Lots of TSM sites attempt to emulate the kind of selective tape retention
described above. However, the goal is usually to emulate the old system's
hit or miss ability to retrieve very old files, not to emulate the old
system's tape handling. Historically, sites trying to emulate selective
tape retention have done TSM incremental backups every day and run
occasional
TSM archives as substitutes for tapes kept for longer than usual periods.
More recently, some sites have experimented with TSM incremental backups
every day and occasional generation of backup sets. If you use TSM
incremental backups at all there are going to be changes in tape handling.
I would suggest that your customer either give up on TSM or give up on
preserving the current tape handling practices.
-Original Message-
From: Marc Levitan [mailto:[EMAIL PROTECTED]]
Sent: Thursday, April 04, 2002 10:16 AM
To: [EMAIL PROTECTED]
Subject: Monthly Backups, ...again!
Has anyone had to defend themselves against the MONTHLY FULL BACKUP kept
for a year scenario???
Business wants a monthly full backup to be kept for a year.
How have people dealt with this issue?
Thanks,
Marc Levitan
Storage Manager
PFPC Global Fund Services
- Forwarded by Marc D Levitan/PFPC/WES/PNC on 04/04/2002 11:15 AM -
Marc D
Levitan To: [EMAIL PROTECTED]
cc:
04/04/2002 Subject: Monthly Backups,
...again!
08:51 AM
A question was brought up while discussing retention policies.
Currently we have the following retentions:
PolicyPolicyMgmt Copy Versions Versions Retain Retain
DomainSet Name Class Group Data DataExtraOnly
NameName NameExists Deleted Versions Version
- - - - ---
COLD ACTIVECOLD STANDARD 215 30
NOVELLACTIVEDIRMC STANDARD301 120 365
NOVELLACTIVESTANDARD STANDARD301 120 365
RECON ACTIVEDIRMC STANDARD363 75 385
RECON ACTIVEMC_RECON STANDARD261 60 365
STANDARD ACTIVEDIRMC STANDARD261 60 365
STANDARD ACTIVESTANDARD STANDARD261 60 365
UNIX ACTIVEMC_UNIX STANDARD301 60 30
I believe that this provides for daily backups for over a month.
There was a request to have the following:
1) Daily backups for a week.
2) Weekly backups for a month.
3) Monthly backups for a year.
I believe we are providing 1 & 2. We are providing daily backups for a
month.
How can I provide monthly backups for a year?
I know that I could take monthly archives, but this would exceed our backup
windows and would increase our resources ( db, tapes, etc.)
Also, I know we could lengthen our retention policies.
Also we could create backup sets. (tons of tapes!)
How are other people handling this?
Thanks,
Marc Levitan
Storage Manager
PFPC Global Fund Services
Re: Monthly Backups, ...again!
The question is what do you hope to restore from it. You will find that a full backup like this applies to a specific application, not the entire company. It is usually a small segment of the corporate data. In the mainframe world with tapes for applications you just created a tape with a specific retention, special backup of that information. There are probably a thousand ways to accomplish this in the TSM open world. The most viable way in the open systems world is to copy the data files to an archive area that is archive/deleted and kept for the appropriate period of time. This way the application has the responsibility to put the data in the location at the appropriate time and the storage management solution now manages that area with a predefined set of criteria that is totally automated. Yes, this requires a little bit of work for the application. But, if they really want data recovery at an application level then this is the most effective way. Everyone, knows the archive data areas then and where to find the archived data. -Original Message- From: Marc Levitan [mailto:[EMAIL PROTECTED]] Sent: Thursday, April 04, 2002 11:16 AM To: [EMAIL PROTECTED] Subject: Monthly Backups, ...again! Has anyone had to defend themselves against the MONTHLY FULL BACKUP kept for a year scenario??? Business wants a monthly full backup to be kept for a year. How have people dealt with this issue? Thanks, Marc Levitan Storage Manager PFPC Global Fund Services - Forwarded by Marc D Levitan/PFPC/WES/PNC on 04/04/2002 11:15 AM - Marc D Levitan To: [EMAIL PROTECTED] cc: 04/04/2002 Subject: Monthly Backups, ...again! 08:51 AM A question was brought up while discussing retention policies. Currently we have the following retentions: PolicyPolicyMgmt Copy Versions Versions Retain Retain DomainSet Name Class Group Data DataExtraOnly NameName NameExists Deleted Versions Version - - - - --- COLD ACTIVECOLD STANDARD 215 30 NOVELLACTIVEDIRMC STANDARD301 120 365 NOVELLACTIVESTANDARD STANDARD301 120 365 RECON ACTIVEDIRMC STANDARD363 75 385 RECON ACTIVEMC_RECON STANDARD261 60 365 STANDARD ACTIVEDIRMC STANDARD261 60 365 STANDARD ACTIVESTANDARD STANDARD261 60 365 UNIX ACTIVEMC_UNIX STANDARD301 60 30 I believe that this provides for daily backups for over a month. There was a request to have the following: 1) Daily backups for a week. 2) Weekly backups for a month. 3) Monthly backups for a year. I believe we are providing 1 & 2. We are providing daily backups for a month. How can I provide monthly backups for a year? I know that I could take monthly archives, but this would exceed our backup windows and would increase our resources ( db, tapes, etc.) Also, I know we could lengthen our retention policies. Also we could create backup sets. (tons of tapes!) How are other people handling this? Thanks, Marc Levitan Storage Manager PFPC Global Fund Services
Re: Monthly Backups, ...again!
My current favorite way to explain Progressive Backups - "TSM updates the full backup incrementally." Get that one across, then explain versioning at the file level, and you've got them! Nick Cassimatis [EMAIL PROTECTED] Today is the tomorrow of yesterday.
Re: Monthly Backups, ...again!
Well, Marc, if you're stuck between the rock and the political hard place, you could create 2 nodes. node node_hybrid Create two schedulers. Use two different optfiles with two different inclexcl lists. Or put the nodes in seperate domains so they can use different default mgmtclasses. Make the schedule for node_hybrid run a normal incremental backup on the first of every month. It's copygroup should be something like PolicyPolicyMgmt Copy Versions Versions Retain Retain DomainSet Name Class Group Data DataExtraOnly NameName NameExists Deleted Versions Version - - - - --- HYBRIDACTIVEMONTHLY STANDARD12 12 365 365 You may want to set the frequency to 28, or some such, just in case someone accidentally tries to kick off a mid-month backup under the hybrid node name. This way only new stuff or stuff that gets changed during the last month will get backed up. Granted, that approximately doubles that node's occupancy, and increases backup processing and network utilization and whatnot once a month, but it'll be better than keeping 12 copies of everything and a lot less effort, processing overhead, and database overhead than most archive scenarios. Alex Paschal Storage Administrator Freightliner, LLC (503) 745-6850 phone/vmail -Original Message- From: Marc Levitan [mailto:[EMAIL PROTECTED]] Sent: Thursday, April 04, 2002 5:51 AM To: [EMAIL PROTECTED] Subject: Monthly Backups, ...again! A question was brought up while discussing retention policies. Currently we have the following retentions: PolicyPolicyMgmt Copy Versions Versions Retain Retain DomainSet Name Class Group Data DataExtraOnly NameName NameExists Deleted Versions Version - - - - --- COLD ACTIVECOLD STANDARD 215 30 NOVELLACTIVEDIRMC STANDARD301 120 365 NOVELLACTIVESTANDARD STANDARD301 120 365 RECON ACTIVEDIRMC STANDARD363 75 385 RECON ACTIVEMC_RECON STANDARD261 60 365 STANDARD ACTIVEDIRMC STANDARD261 60 365 STANDARD ACTIVESTANDARD STANDARD261 60 365 UNIX ACTIVEMC_UNIX STANDARD301 60 30 I believe that this provides for daily backups for over a month. There was a request to have the following: 1) Daily backups for a week. 2) Weekly backups for a month. 3) Monthly backups for a year. I believe we are providing 1 & 2. We are providing daily backups for a month. How can I provide monthly backups for a year? I know that I could take monthly archives, but this would exceed our backup windows and would increase our resources ( db, tapes, etc.) Also, I know we could lengthen our retention policies. Also we could create backup sets. (tons of tapes!) How are other people handling this? Thanks, Marc Levitan Storage Manager PFPC Global Fund Services
Re: Monthly Backups, ...again!
There is a good reason it keeps coming up: legitimate business
requirements.
The suits (auditors, IRS, corp counsel, HIPAA, etc) demand to be able to
be able to reproduce any datum at given intervals for given durations.
Most often, that translates to restoring files that may change every day
to "month end" state for somewhere between 1 and 7 years. Sometime they
can identify the kinds of data they want, but it is expensive to
accurately identify the list of all files/directories required, so usually
you get a vague wave to "save everything". And of course, it's their
data, not yours, they have a right to keep as much as they want. Telling
them that TSM doesn't support their requirement just invites other
software vendors in the door since *they* handle this particular
requirement with ease (on paper).
The number of days you can reasonably keep in an incremental backup
usually doesn't extend to forever. Archives sometimes don't cut it,
either in their traditional form or the instant form. You can't stand to
move that much data or use that many tapes - that's why you went
incremental forever in the first place. I really just to do some
operation that marks the current active version with a longer guaranteed
retention, without changing the retention of anything else.
I don't want to restart the perennial discussion of truly long term
archival storage. It's reasonable to expect a backup system to maintain
internal compatibility for 7 years, and there are techniques for migrating
the data to newer media.
Just my 5 cents worth (inflation).
_
William Mansfield
Senior Consultant
Solution Technology, Inc
"Mr. Lindsay Morris" <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
04/04/2002 10:04 AM
Please respond to lmorris
To: [EMAIL PROTECTED]
cc:
Subject:Re: Monthly Backups, ...again!
This keeps coming up. It's the hardest thing about TSM, to sell users on
the way it works.
Tivoli's Storage Vision whitepaper has a comparison of the benefits you
get
by NOT using this Grandfather-father-son technique, but I wish somebody at
Tivoli would come up with some better assistance to help us sell the
incremental-forever -ooops, progressive backup methodolgy - to non-techie
users. (Maybe it's there and I just don't know where to find it...?)
I think Kelly Lipp has a good article on archiving and when it's sensible
-
maybe he'll post that link here again.
Also, maybe some users have specific oddball scenarios they have run into
that require surprising policy settings. It would be interesting to hear
about those. Like, the user who goes on vacation for two weeks, and
manages
to trash here email file the day she leaves, doesn't notice it, Lotus
touches the damaged file every day so it gets backed up again, and they
don't keep 14 versions, so she gets back and the only good version (15
days
old) has rolled off (expired).
-
Mr. Lindsay Morris
CEO, Servergraph
www.servergraph.com
859-253-8000 ofc
425-988-8478 fax
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of
> Marc Levitan
> Sent: Thursday, April 04, 2002 8:51 AM
> To: [EMAIL PROTECTED]
> Subject: Monthly Backups, ...again!
>
>
> A question was brought up while discussing retention policies.
>
> Currently we have the following retentions:
>
> PolicyPolicyMgmt Copy Versions Versions Retain
Retain
> DomainSet Name Class Group Data DataExtra Only
> NameName NameExists Deleted Versions
Version
> - - - -
---
> COLD ACTIVECOLD STANDARD 215 30
>
> NOVELLACTIVEDIRMC STANDARD301 120 365
> NOVELLACTIVESTANDARD STANDARD301 120 365
>
> RECON ACTIVEDIRMC STANDARD363 75 385
> RECON ACTIVEMC_RECON STANDARD261 60 365
>
> STANDARD ACTIVEDIRMC STANDARD261 60 365
> STANDARD ACTIVESTANDARD STANDARD261 60 365
>
>
> UNIX ACTIVEMC_UNIX STANDARD301 60 30
>
>
> I believe that this provides for daily backups for over a month.
>
> There was a request to have the following:
> 1) Daily backups for a week.
> 2) Weekly backups for a month.
> 3) Monthly backups for a year.
>
> I believe we are providing 1 & 2. We are providing daily backups for a
> month.
>
> How can I provide monthly backups for a year?
> I know that I could take monthly archives, but
Re: Monthly Backups, ...again!
Not a problem. I look at the space required for the database entries and the amount of media required (and it *will* be copied for off-site, or they really don't need it and won't get it!) and let them know what the cost will be. Then we set up an archive script, with a management class that has a 395 day retention, and archive the data. I refuse to jump through hoops to get retain only/retain extra/ to keep data for some number of days or years -- if the retention requirements come to me in days/weeks/months/years, it's an archive. And I support retention for 40 days, 110 days, 395 days, and 'x' years (2 through 7) at x * 366 + 30 days. I also let the requestor know that the process for extending the retention longer than the original archive is to take an outage, archive the current data (or rename the file systems), retrieve the data in question, and re-archive with the proper management class. Tom Kauffman NIBCO, Inc > -Original Message- > From: Marc Levitan [mailto:[EMAIL PROTECTED]] > Sent: Thursday, April 04, 2002 11:16 AM > To: [EMAIL PROTECTED] > Subject: Monthly Backups, ...again! > > > Has anyone had to defend themselves against the MONTHLY FULL > BACKUP kept > for a year scenario??? > Business wants a monthly full backup to be kept for a year. > How have people dealt with this issue? > > Thanks, > Marc Levitan > Storage Manager > PFPC Global Fund Services > > > - Forwarded by Marc D Levitan/PFPC/WES/PNC on 04/04/2002 > 11:15 AM - > > Marc D > Levitan To: [EMAIL PROTECTED] > cc: > 04/04/2002 Subject: Monthly > Backups, ...again! > 08:51 AM > > > > > > A question was brought up while discussing retention policies. > > Currently we have the following retentions: > > PolicyPolicyMgmt Copy Versions Versions > Retain Retain > DomainSet Name Class Group Data Data > ExtraOnly > NameName NameExists Deleted > Versions Version > - - - - > --- > COLD ACTIVECOLD STANDARD 21 >5 30 > > NOVELLACTIVEDIRMC STANDARD301 > 120 365 > NOVELLACTIVESTANDARD STANDARD301 > 120 365 > > RECON ACTIVEDIRMC STANDARD363 > 75 385 > RECON ACTIVEMC_RECON STANDARD261 > 60 365 > > STANDARD ACTIVEDIRMC STANDARD261 > 60 365 > STANDARD ACTIVESTANDARD STANDARD261 > 60 365 > > > UNIX ACTIVEMC_UNIX STANDARD301 > 60 30 > > > I believe that this provides for daily backups for over a month. > > There was a request to have the following: > 1) Daily backups for a week. > 2) Weekly backups for a month. > 3) Monthly backups for a year. > > I believe we are providing 1 & 2. We are providing daily > backups for a > month. > > How can I provide monthly backups for a year? > I know that I could take monthly archives, but this would > exceed our backup > windows and would increase our resources ( db, tapes, etc.) > Also, I know we could lengthen our retention policies. > Also we could create backup sets. (tons of tapes!) > > How are other people handling this? > > Thanks, > > > Marc Levitan > Storage Manager > PFPC Global Fund Services >
Re: Monthly Backups, ...again!
I handle this requirement with a SECOND client on each machine which does
an INCREMENTAL backup bound to a management class with the appropriate
retention time. This eliminates the resending of static data. This is
easily accomplished with a second dsm.opt file and a second server stanza.
Orville L. Lantto
Datatrend Technologies, Inc. (http://www.datatrend.com)
121 Cheshire Lane #700
Minnetonka, MN 55305
Email: [EMAIL PROTECTED]
V: 952-931-1203
F: 952-931-1293
C: 612-770-9166
"Cook, Dwight E" <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
04/04/02 11:01 AM
Please respond to "ADSM: Dist Stor Manager"
To: [EMAIL PROTECTED]
cc:
Subject:Re: Monthly Backups, ...again!
ARCHIVE !
I tell clients, if they HAVE to have data for a SPECIFIC period of time,
archive it into a management class with the required retention period.
I try to only give them 1 3, 6, & 12 month management classes...
anything longer than that and I make them archive to a specially
registered
node name, I export that data, send the export tapes offsite and purge
that
specific info out of my TSM data base ;-)
Dwight E. Cook
Software Application Engineer III
Science Applications International Corporation
509 S. Boston Ave. Suit 220
Tulsa, Oklahoma 74103-4606
Office (918) 732-7109
-Original Message-
From: Marc Levitan [mailto:[EMAIL PROTECTED]]
Sent: Thursday, April 04, 2002 10:16 AM
To: [EMAIL PROTECTED]
Subject: Monthly Backups, ...again!
Has anyone had to defend themselves against the MONTHLY FULL BACKUP kept
for a year scenario???
Business wants a monthly full backup to be kept for a year.
How have people dealt with this issue?
Thanks,
Marc Levitan
Storage Manager
PFPC Global Fund Services
- Forwarded by Marc D Levitan/PFPC/WES/PNC on 04/04/2002 11:15 AM
-
Marc D
Levitan To: [EMAIL PROTECTED]
cc:
04/04/2002 Subject: Monthly Backups,
...again!
08:51 AM
A question was brought up while discussing retention policies.
Currently we have the following retentions:
PolicyPolicyMgmt Copy Versions Versions Retain Retain
DomainSet Name Class Group Data DataExtraOnly
NameName NameExists Deleted Versions Version
- - - - ---
COLD ACTIVECOLD STANDARD 215 30
NOVELLACTIVEDIRMC STANDARD301 120 365
NOVELLACTIVESTANDARD STANDARD301 120 365
RECON ACTIVEDIRMC STANDARD363 75 385
RECON ACTIVEMC_RECON STANDARD261 60 365
STANDARD ACTIVEDIRMC STANDARD261 60 365
STANDARD ACTIVESTANDARD STANDARD261 60 365
UNIX ACTIVEMC_UNIX STANDARD301 60 30
I believe that this provides for daily backups for over a month.
There was a request to have the following:
1) Daily backups for a week.
2) Weekly backups for a month.
3) Monthly backups for a year.
I believe we are providing 1 & 2. We are providing daily backups for a
month.
How can I provide monthly backups for a year?
I know that I could take monthly archives, but this would exceed our
backup
windows and would increase our resources ( db, tapes, etc.)
Also, I know we could lengthen our retention policies.
Also we could create backup sets. (tons of tapes!)
How are other people handling this?
Thanks,
Marc Levitan
Storage Manager
PFPC Global Fund Services
Re: Monthly Backups, ...again!
If that is what you need, after the last incremental for the month, build a backupset for the servers that have this requirement. This should give the business what they need. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]] On Behalf Of Marc Levitan Sent: Thursday, April 04, 2002 11:16 AM To: [EMAIL PROTECTED] Subject: Monthly Backups, ...again! Has anyone had to defend themselves against the MONTHLY FULL BACKUP kept for a year scenario??? Business wants a monthly full backup to be kept for a year. How have people dealt with this issue? Thanks, Marc Levitan Storage Manager PFPC Global Fund Services - Forwarded by Marc D Levitan/PFPC/WES/PNC on 04/04/2002 11:15 AM - Marc D Levitan To: [EMAIL PROTECTED] cc: 04/04/2002 Subject: Monthly Backups, ...again! 08:51 AM A question was brought up while discussing retention policies. Currently we have the following retentions: PolicyPolicyMgmt Copy Versions Versions Retain Retain DomainSet Name Class Group Data DataExtra Only NameName NameExists Deleted Versions Version - - - - --- COLD ACTIVECOLD STANDARD 215 30 NOVELLACTIVEDIRMC STANDARD301 120 365 NOVELLACTIVESTANDARD STANDARD301 120 365 RECON ACTIVEDIRMC STANDARD363 75 385 RECON ACTIVEMC_RECON STANDARD261 60 365 STANDARD ACTIVEDIRMC STANDARD261 60 365 STANDARD ACTIVESTANDARD STANDARD261 60 365 UNIX ACTIVEMC_UNIX STANDARD301 60 30 I believe that this provides for daily backups for over a month. There was a request to have the following: 1) Daily backups for a week. 2) Weekly backups for a month. 3) Monthly backups for a year. I believe we are providing 1 & 2. We are providing daily backups for a month. How can I provide monthly backups for a year? I know that I could take monthly archives, but this would exceed our backup windows and would increase our resources ( db, tapes, etc.) Also, I know we could lengthen our retention policies. Also we could create backup sets. (tons of tapes!) How are other people handling this? Thanks, Marc Levitan Storage Manager PFPC Global Fund Services
Re: Monthly Backups, ...again!
I know I am flogging a dead horse here. In order to define your backup/restore policies you must focus on the businesses RESTORE REQUIREMENTS and then determine what backups/archives are required to meet those requirements. Remember the only reason (outside of appeasing the political requirements of those who do not understand) for doing backups is to provide the ability to restore. DO NOT focus on how many backups are required until you understand the RESTORE REQUIREMENTS. This seems to be the only way that I manage to get around the archaic way of thinking when it comes to doing backups, when required. My two cents worth. Jim Taylor -Original Message- From: Marc Levitan [mailto:[EMAIL PROTECTED]] Sent: Thursday, April 04, 2002 11:16 AM To: [EMAIL PROTECTED] Subject: Monthly Backups, ...again! Has anyone had to defend themselves against the MONTHLY FULL BACKUP kept for a year scenario??? Business wants a monthly full backup to be kept for a year. How have people dealt with this issue? Thanks, Marc Levitan Storage Manager PFPC Global Fund Services - Forwarded by Marc D Levitan/PFPC/WES/PNC on 04/04/2002 11:15 AM - Marc D Levitan To: [EMAIL PROTECTED] cc: 04/04/2002 Subject: Monthly Backups, ...again! 08:51 AM A question was brought up while discussing retention policies. Currently we have the following retentions: PolicyPolicyMgmt Copy Versions Versions Retain Retain DomainSet Name Class Group Data DataExtraOnly NameName NameExists Deleted Versions Version - - - - --- COLD ACTIVECOLD STANDARD 215 30 NOVELLACTIVEDIRMC STANDARD301 120 365 NOVELLACTIVESTANDARD STANDARD301 120 365 RECON ACTIVEDIRMC STANDARD363 75 385 RECON ACTIVEMC_RECON STANDARD261 60 365 STANDARD ACTIVEDIRMC STANDARD261 60 365 STANDARD ACTIVESTANDARD STANDARD261 60 365 UNIX ACTIVEMC_UNIX STANDARD301 60 30 I believe that this provides for daily backups for over a month. There was a request to have the following: 1) Daily backups for a week. 2) Weekly backups for a month. 3) Monthly backups for a year. I believe we are providing 1 & 2. We are providing daily backups for a month. How can I provide monthly backups for a year? I know that I could take monthly archives, but this would exceed our backup windows and would increase our resources ( db, tapes, etc.) Also, I know we could lengthen our retention policies. Also we could create backup sets. (tons of tapes!) How are other people handling this? Thanks, Marc Levitan Storage Manager PFPC Global Fund Services
Re: Monthly Backups, ...again!
ARCHIVE ! I tell clients, if they HAVE to have data for a SPECIFIC period of time, archive it into a management class with the required retention period. I try to only give them 1 3, 6, & 12 month management classes... anything longer than that and I make them archive to a specially registered node name, I export that data, send the export tapes offsite and purge that specific info out of my TSM data base ;-) Dwight E. Cook Software Application Engineer III Science Applications International Corporation 509 S. Boston Ave. Suit 220 Tulsa, Oklahoma 74103-4606 Office (918) 732-7109 -Original Message- From: Marc Levitan [mailto:[EMAIL PROTECTED]] Sent: Thursday, April 04, 2002 10:16 AM To: [EMAIL PROTECTED] Subject: Monthly Backups, ...again! Has anyone had to defend themselves against the MONTHLY FULL BACKUP kept for a year scenario??? Business wants a monthly full backup to be kept for a year. How have people dealt with this issue? Thanks, Marc Levitan Storage Manager PFPC Global Fund Services - Forwarded by Marc D Levitan/PFPC/WES/PNC on 04/04/2002 11:15 AM - Marc D Levitan To: [EMAIL PROTECTED] cc: 04/04/2002 Subject: Monthly Backups, ...again! 08:51 AM A question was brought up while discussing retention policies. Currently we have the following retentions: PolicyPolicyMgmt Copy Versions Versions Retain Retain DomainSet Name Class Group Data DataExtraOnly NameName NameExists Deleted Versions Version - - - - --- COLD ACTIVECOLD STANDARD 215 30 NOVELLACTIVEDIRMC STANDARD301 120 365 NOVELLACTIVESTANDARD STANDARD301 120 365 RECON ACTIVEDIRMC STANDARD363 75 385 RECON ACTIVEMC_RECON STANDARD261 60 365 STANDARD ACTIVEDIRMC STANDARD261 60 365 STANDARD ACTIVESTANDARD STANDARD261 60 365 UNIX ACTIVEMC_UNIX STANDARD301 60 30 I believe that this provides for daily backups for over a month. There was a request to have the following: 1) Daily backups for a week. 2) Weekly backups for a month. 3) Monthly backups for a year. I believe we are providing 1 & 2. We are providing daily backups for a month. How can I provide monthly backups for a year? I know that I could take monthly archives, but this would exceed our backup windows and would increase our resources ( db, tapes, etc.) Also, I know we could lengthen our retention policies. Also we could create backup sets. (tons of tapes!) How are other people handling this? Thanks, Marc Levitan Storage Manager PFPC Global Fund Services
Monthly Backups, ...again!
Has anyone had to defend themselves against the MONTHLY FULL BACKUP kept for a year scenario??? Business wants a monthly full backup to be kept for a year. How have people dealt with this issue? Thanks, Marc Levitan Storage Manager PFPC Global Fund Services - Forwarded by Marc D Levitan/PFPC/WES/PNC on 04/04/2002 11:15 AM - Marc D Levitan To: [EMAIL PROTECTED] cc: 04/04/2002 Subject: Monthly Backups, ...again! 08:51 AM A question was brought up while discussing retention policies. Currently we have the following retentions: PolicyPolicyMgmt Copy Versions Versions Retain Retain DomainSet Name Class Group Data DataExtraOnly NameName NameExists Deleted Versions Version - - - - --- COLD ACTIVECOLD STANDARD 215 30 NOVELLACTIVEDIRMC STANDARD301 120 365 NOVELLACTIVESTANDARD STANDARD301 120 365 RECON ACTIVEDIRMC STANDARD363 75 385 RECON ACTIVEMC_RECON STANDARD261 60 365 STANDARD ACTIVEDIRMC STANDARD261 60 365 STANDARD ACTIVESTANDARD STANDARD261 60 365 UNIX ACTIVEMC_UNIX STANDARD301 60 30 I believe that this provides for daily backups for over a month. There was a request to have the following: 1) Daily backups for a week. 2) Weekly backups for a month. 3) Monthly backups for a year. I believe we are providing 1 & 2. We are providing daily backups for a month. How can I provide monthly backups for a year? I know that I could take monthly archives, but this would exceed our backup windows and would increase our resources ( db, tapes, etc.) Also, I know we could lengthen our retention policies. Also we could create backup sets. (tons of tapes!) How are other people handling this? Thanks, Marc Levitan Storage Manager PFPC Global Fund Services
Re: Monthly Backups, ...again!
This keeps coming up. It's the hardest thing about TSM, to sell users on the way it works. Tivoli's Storage Vision whitepaper has a comparison of the benefits you get by NOT using this Grandfather-father-son technique, but I wish somebody at Tivoli would come up with some better assistance to help us sell the incremental-forever -ooops, progressive backup methodolgy - to non-techie users. (Maybe it's there and I just don't know where to find it...?) I think Kelly Lipp has a good article on archiving and when it's sensible - maybe he'll post that link here again. Also, maybe some users have specific oddball scenarios they have run into that require surprising policy settings. It would be interesting to hear about those. Like, the user who goes on vacation for two weeks, and manages to trash here email file the day she leaves, doesn't notice it, Lotus touches the damaged file every day so it gets backed up again, and they don't keep 14 versions, so she gets back and the only good version (15 days old) has rolled off (expired). - Mr. Lindsay Morris CEO, Servergraph www.servergraph.com 859-253-8000 ofc 425-988-8478 fax > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of > Marc Levitan > Sent: Thursday, April 04, 2002 8:51 AM > To: [EMAIL PROTECTED] > Subject: Monthly Backups, ...again! > > > A question was brought up while discussing retention policies. > > Currently we have the following retentions: > > PolicyPolicyMgmt Copy Versions Versions Retain Retain > DomainSet Name Class Group Data DataExtraOnly > NameName NameExists Deleted Versions Version > - - - - --- > COLD ACTIVECOLD STANDARD 215 30 > > NOVELLACTIVEDIRMC STANDARD301 120 365 > NOVELLACTIVESTANDARD STANDARD301 120 365 > > RECON ACTIVEDIRMC STANDARD363 75 385 > RECON ACTIVEMC_RECON STANDARD261 60 365 > > STANDARD ACTIVEDIRMC STANDARD261 60 365 > STANDARD ACTIVESTANDARD STANDARD261 60 365 > > > UNIX ACTIVEMC_UNIX STANDARD301 60 30 > > > I believe that this provides for daily backups for over a month. > > There was a request to have the following: > 1) Daily backups for a week. > 2) Weekly backups for a month. > 3) Monthly backups for a year. > > I believe we are providing 1 & 2. We are providing daily backups for a > month. > > How can I provide monthly backups for a year? > I know that I could take monthly archives, but this would exceed > our backup > windows and would increase our resources ( db, tapes, etc.) > Also, I know we could lengthen our retention policies. > Also we could create backup sets. (tons of tapes!) > > How are other people handling this? > > Thanks, > > > Marc Levitan > Storage Manager > PFPC Global Fund Services >
Re: Monthly Backups, ...again!
Do you use copypools? How much do you want to spend? How fast do you need to restore from a monthly? How about this scenario; it is just an idea, I don't actually do it - make a 2nd copypool that stays on site. Don't do any reclaims of it. At the start of the month do a db snapshot backup and retain the tape for a year. When you need to do a restore from a monthly backup, restore the appropriate monthly db snapshot to a new instance of TSM. See the admin reference appendix for 'restore a db without a volumehistory file'. In the restored instance, mark all the tapes in the primary pool destroyed and the diskpool too.. Also be sure all the tapes in the offsite copy pool are marked offsite or destroyed too. Now a client file restore should access the onsite copypool. The file should be there since you have not done any reclaims of this pool. A year into this process you can start doing 'move data recon=yes' on the onsite copypool volumes, selecting volumes written more than a year ago. The cost is mostly media if you have enough processor to make the 2nd copypool. Your database will be bigger and you will need extra disk space for the db restore. What do you think? Hope this helps, Bill Colwell At 08:51 AM 4/4/2002 -0500, you wrote: >A question was brought up while discussing retention policies. > >Currently we have the following retentions: > >PolicyPolicyMgmt Copy Versions Versions Retain Retain >DomainSet Name Class Group Data DataExtraOnly >NameName NameExists Deleted Versions Version >- - - - --- >COLD ACTIVECOLD STANDARD 215 30 > >NOVELLACTIVEDIRMC STANDARD301 120 365 >NOVELLACTIVESTANDARD STANDARD301 120 365 > >RECON ACTIVEDIRMC STANDARD363 75 385 >RECON ACTIVEMC_RECON STANDARD261 60 365 > >STANDARD ACTIVEDIRMC STANDARD261 60 365 >STANDARD ACTIVESTANDARD STANDARD261 60 365 > > >UNIX ACTIVEMC_UNIX STANDARD301 60 30 > > >I believe that this provides for daily backups for over a month. > >There was a request to have the following: >1) Daily backups for a week. >2) Weekly backups for a month. >3) Monthly backups for a year. > >I believe we are providing 1 & 2. We are providing daily backups for a >month. > >How can I provide monthly backups for a year? >I know that I could take monthly archives, but this would exceed our backup >windows and would increase our resources ( db, tapes, etc.) >Also, I know we could lengthen our retention policies. >Also we could create backup sets. (tons of tapes!) > >How are other people handling this? > >Thanks, > > >Marc Levitan >Storage Manager >PFPC Global Fund Services -- Bill Colwell C. S. Draper Lab Cambridge Ma.
Monthly Backups, ...again!
A question was brought up while discussing retention policies. Currently we have the following retentions: PolicyPolicyMgmt Copy Versions Versions Retain Retain DomainSet Name Class Group Data DataExtraOnly NameName NameExists Deleted Versions Version - - - - --- COLD ACTIVECOLD STANDARD 215 30 NOVELLACTIVEDIRMC STANDARD301 120 365 NOVELLACTIVESTANDARD STANDARD301 120 365 RECON ACTIVEDIRMC STANDARD363 75 385 RECON ACTIVEMC_RECON STANDARD261 60 365 STANDARD ACTIVEDIRMC STANDARD261 60 365 STANDARD ACTIVESTANDARD STANDARD261 60 365 UNIX ACTIVEMC_UNIX STANDARD301 60 30 I believe that this provides for daily backups for over a month. There was a request to have the following: 1) Daily backups for a week. 2) Weekly backups for a month. 3) Monthly backups for a year. I believe we are providing 1 & 2. We are providing daily backups for a month. How can I provide monthly backups for a year? I know that I could take monthly archives, but this would exceed our backup windows and would increase our resources ( db, tapes, etc.) Also, I know we could lengthen our retention policies. Also we could create backup sets. (tons of tapes!) How are other people handling this? Thanks, Marc Levitan Storage Manager PFPC Global Fund Services
