Re: [Veritas-bu] Archive list

[Wayne T Smith]

Good points and summary, Bob.

My argument wasn't that archives are less restorable, especially if
they were covered by scheduled backups before the archive.  They are
probably in as safe a place as the rest of the backups.

What I meant was that without the live copy, my chances of losing the
data altogether are greater.  I used to have three copies (live and
traditional onsite and offsite backups), but now have, at best, only
two.   Two "should be" more than enough, but its not as good as three.

I'd argue that unless the backup administrator and data owner
understand NetBackup "archive" and their own NetBackup implementation,
data loss can occur.

For example, a few weeks ago I had a data storage failure moments
after a backup "successfully" finished and before the data could be
replicated to a second backup storage.

If my data owner had data new to the backup system and  the
"successful" backup job was instead an archive, that data would have
been lost.

I would feel better about NetBackup archive if the live data deletion
could be made to occur once the backup system "successfully"
replicated, but I still contend the organization needs to realize that
if archive is stored like backup, once archived, the data is in one
less place than before.

My thanks to all contributors, too.

Cheers, Wayne

On Sun, Mar 11, 2012 at 3:54 AM, bob944  wrote, in part:
>
> > I think the place for NBU "archive" is quite limited and is not a
> > reasonable solution where one is expected to be able to restore the
> > data to their usable place/form.
>
> Agreed that its place is limited but why is it any more unreasonable
> to "restore the data to their usable place/form" than if the data had
> been backed up by a full/diff/incr schedule and then blown away by the
> user?  You have the same storage destinations, volume pools, copies,
> retentions, ownership, multiplexing, ... as any other backup--all
> under the admin's complete control.  Why should I be less able to
> restore the data "to their usable place/form" than from, say, the
> previous night's scheduled full?
___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

Re: [Veritas-bu] Archive list

[bob944]

Haven't seen this much interest in archives (in the NBU sense) in
years.  GOOD discussion.

[confession:]  Hi.  My name's bob944 and I've used user archives.

Wow.  This is the most press UARC has received in ages.

First, every person who commented on this made sense and appears to
understand the way NBU archives work.

Second, an issue:
o  Not telling the user community about archives is like not telling
them the rm or del commands exist.  Users--and admins--make mistakes,
but my personal High & Mighty Pedestal isn't quite high enough to
decide what they can't know about.

In the following, I'm assuming that 
o  UBAKs and UARCs are understood by the admin and the owner of the
data:  user specifies the data; upon successful archive, the data are
removed
o  I and the users decided where and for how long the data are be
preserved

Re: David Rock's follow-up:
> Backup admins are generally in the business of preserving data
> for recovery, not removal of files from systems.

I agree, yet don't see a conflict here.  IMO, the admin _does_
preserves the data--automatically at it's very last (that may be
important) state--before the user blows it away.  The user has made
sure "we" backed it up before it's deleted.  He could have just
deleted it himself.  Why is that not a win?

If we back up /ora/db/our_2011_customer_masterfile.dbf, we do it so
that we can recover it after some bonehead mistake OR DELIBERATE USER
ACTION ("the backup guys have backed up the DB; let's blow it away and
go with the new schema") removes data that is needed today or a year
from now.  There's no intrinsic difference when we, and the user, set
up an archive.  The user can already 'rm
our_2011_customer_masterfile.dbf' without any archive in scope, call
up and need it restored.  Same thing for a deliberate decision by the
user that they no longer need it--but just in case, would like the
last version backed up before their schema change.  To Mr Rock's
point, we _have_ preserved the data so that the user can remove it
with assurance that it can be recovered within the parameters agreed
to--the same as for backups.  I see no difference between a UARC and a
morning phone call of "did you back up that old customer master file?"
| "yes" | "Okay, we're blowing it away." 

Unless one has a) incorrectly instructed the user community on what a
UARC does, b) set the UARC parameters to something inferior to what
the user expects  or c) treats UARCs as sacrificial data, I
don't/haven't seen an issue:  we're doing exactly what we do for
scheduled backups--preserving the data to agreed-upon standards.

Regarding the archive policy (I'll still call it that, assuming that
the admin and the user know "the user runs a user backup, specifying
what gets backed up and, if it's an archive and successful,
subsequently deleted"), I've only set this up in production a couple
of times in the last 15 years.  My favorite was in the late '90s with
NBU 3.1 or 3.2.  A user group had a finance application in which
month-end created an elaborate set of directories, subdirectories
(thousands) and files (hundreds of thousands).  The app left the files
in place, leaving it to the user to decide when the data were of no
value.  None of the users (or perhaps, "their management") was keen on
using a command line or GUI to recursively remove thousands of
directories and the files therein, and the space used was a big issue
in those days.  They used the app's data for a day or two, then their
business process used a UARC schedule on (IIRC) a unique policy to
back it up "just in case" and delete it.  I've also set up UARC
schedules for groups that were leaving the corporate HQ's backup
umbrella so they could get tapes of their data to be recovered in
their new "home."

[Rock:]
> Basically, what you are trying to do is something that NBU
> is not meant to do.  The expected behavior for Archives is
> that they are managed by the client admin, not the backup
>admin.  Archive jobs get a filelist from the user, run the
> archive, and 

[if and only if the backups were 100% successful,]

> delete the files when they are done, so the user would
> already know what files were archived because they would
> have supplied the list to the command in the first place.

Whether the original poster understood this or not, IMO it's an
excellent clarification of how UBAKs (without the deletion) and UARCs
work.  It's scary or unknown territory to many admins and most all
users.

> From: Kevin Holtz
> All is true but I wouldn't say it's something NBU was not designed
> to do.  It was made difficult for a reason rather than easy due to
> the nature of the outcome e.g. Archiving.  This is very common
> practice for DBA's.  Yes, this all needs to be scripted, blat etc.
> and you will need admins to help if you don't have access to the
> local systems.

What he said, except the admin isn't helping me, I'm helping them by
providing a user tool.  Though I think Rock was commenting on the
"policy" a

Re: [Veritas-bu] Archive list

[Wayne T Smith]

When I consider protecting data, I'm typically thinking of keeping them in
multiple places that are as independent as possible.

If I were to use NBU "archive", I would be basically chopping off one of
the places where the data is stored (its original, live home).

I think the place for NBU "archive" is quite limited and is not a
reasonable solution where one is expected to be able to restore the data to
their usable place/form.  For the case where "it would be nice, but not
critical, to restore this data sometime in the next x months", NBU
"archive" might be an easy solution.   I've never encountered that case.
Is it better than writing them to DVDs? Maybe.

Cheers, Wayne

On Fri, Mar 9, 2012 at 9:04 AM, David Rock  wrote, in
part:

> * Kevin Holtz  [2012-03-09 07:47]:
> > All is true but I wouldn't say it's something NBU was not designed to
> > do.  It was made difficult for a reason rather than easy due to the
> > nature of the outcome e.g. Archiving.  This is very common practice
> > for DBA's.  Yes, this all needs to be scripted, blat etc. and you will
> > need admins to help if you don't have access to the local systems.
>
> By "not designed to do" I specifically mean that NBU is not designed to
> have the backup admins be responsible for running archive jobs.
> Archives are inherently dangerous and as such, only the owner of the
> files being archived is really intended to be the one to determine of
> archiving is ok.
>
> Backup admins are generally in the business of preserving data for
> recovery, not removal of files from systems.
>
___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

Re: [Veritas-bu] Archive list

[David Rock]

* Kevin Holtz  [2012-03-09 07:47]:
> All is true but I wouldn't say it's something NBU was not designed to
> do.  It was made difficult for a reason rather than easy due to the
> nature of the outcome e.g. Archiving.  This is very common practice
> for DBA's.  Yes, this all needs to be scripted, blat etc. and you will
> need admins to help if you don't have access to the local systems. 

By "not designed to do" I specifically mean that NBU is not designed to
have the backup admins be responsible for running archive jobs.
Archives are inherently dangerous and as such, only the owner of the
files being archived is really intended to be the one to determine of
archiving is ok.

Backup admins are generally in the business of preserving data for
recovery, not removal of files from systems.

-- 
David Rock
da...@graniteweb.com


pgpGwem0CmOpv.pgp
Description: PGP signature
___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

Re: [Veritas-bu] Archive list

[Kevin Holtz]

All is true but I wouldn't say it's something NBU was not designed to do.  It 
was made difficult for a reason rather than easy due to the nature of the 
outcome e.g. Archiving.  This is very common practice for DBA's.  Yes, this all 
needs to be scripted, blat etc. and you will need admins to help if you don't 
have access to the local systems. 

Regards...

Sent from my mobile

On Mar 8, 2012, at 10:28 PM, David Rock  wrote:

> * Dennis Peacock  [2012-03-08 07:14]:
>> Patrick,
>> I'm not a windows guy, so I'm trying to see what I can do without
>> having to find someone that can write a "windows script" to do this.
>> 
>> What I'd prefer to be done is for me to setup a backup policy in NBU
>> that is an Archive type policy. Point that policy to
>> D:\Archive_Data\LZ and schedule the backup to run every night at 22:00
>> hours. When the backup finishes successfully, it would remove the
>> files and then run the bpflist or whatever to generate a list of what
>> was backed up and email that to the customer.
>> 
>> That's what I'd prefer to do, but I'm lacking in knowledge in this
>> windows stuff as I've been living in the Unix/Linux world for over 30
>> years.  [Wink]
> 
> Well, 
> 
> First of all, there is no such thing as an "Archive" Policy type.  The
> closest thing you would have is a Policy type of "MS-Windows" with a
> schedule type of "User Archive".  Archives are designed to be User
> initiated, not Scheduled from the Master Server.
> 
> As for notifications, especially because of the nature of Archives, that
> is something that is going to pretty much _have_ to be a Windows script
> on the client system.  You would have the logic in the clientside script
> to manage all of that.  There are a couple likely issues:
> 
> 1. The Windows server doesn't have an MTA to send email with (typically
> there is no sendmail or equivalent on Windows, so that would need to be
> added)
> 
> 2. It's pretty much guaranteed that this won't be something that will be
> managed by you, it will be the Windows admins (although you would be
> responsible for making the User Archive schedule available to them).
> 
> You _might_ be able to come up with some insane scheme like using a
> scheduled backup that touches the client system, causes the clientside
> script to run, and then uses bpend_notify once it's done to do all the
> other heavy lifting.  That still won't get you away from Windows
> scripting.
> 
> Basically, what you are trying to do is something that NBU is not meant
> to do.  The expected behavior for Archives is that they are managed by
> the client admin, not the backup admin.  Archive jobs get a filelist
> from the user, run the archive, and delete the files when they are done,
> so the user would already know what files were archived because they
> would have supplied the list to the command in the first place.
> 
> You would, at least, be able to query the backup info from the master
> the following day and report what was backed up, but that's about it.
> 
> -- 
> David Rock
> da...@graniteweb.com
> ___
> Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

Re: [Veritas-bu] Archive list

[Lightner, Jeff]

I gather lots of Windows admins (including ours) use BLAT for email from 
Windows for NBU.

The link below talks about how to set BLAT up:
http://www.symantec.com/business/support/index?page=content&id=TECH24110
It's not something I've done but I've heard enough Windows Admins mention it to 
believe it is something that works.





-Original Message-
From: veritas-bu-boun...@mailman.eng.auburn.edu 
[mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of David Rock
Sent: Thursday, March 08, 2012 10:28 PM
To: veritas-bu@mailman.eng.auburn.edu
Subject: Re: [Veritas-bu] Archive list

* Dennis Peacock  [2012-03-08 07:14]:
> Patrick,
> I'm not a windows guy, so I'm trying to see what I can do without
> having to find someone that can write a "windows script" to do this.
>
> What I'd prefer to be done is for me to setup a backup policy in NBU
> that is an Archive type policy. Point that policy to
> D:\Archive_Data\LZ and schedule the backup to run every night at 22:00
> hours. When the backup finishes successfully, it would remove the
> files and then run the bpflist or whatever to generate a list of what
> was backed up and email that to the customer.
>
> That's what I'd prefer to do, but I'm lacking in knowledge in this
> windows stuff as I've been living in the Unix/Linux world for over 30
> years.  [Wink]

Well,

First of all, there is no such thing as an "Archive" Policy type.  The
closest thing you would have is a Policy type of "MS-Windows" with a
schedule type of "User Archive".  Archives are designed to be User
initiated, not Scheduled from the Master Server.

As for notifications, especially because of the nature of Archives, that
is something that is going to pretty much _have_ to be a Windows script
on the client system.  You would have the logic in the clientside script
to manage all of that.  There are a couple likely issues:

1. The Windows server doesn't have an MTA to send email with (typically
there is no sendmail or equivalent on Windows, so that would need to be
added)

2. It's pretty much guaranteed that this won't be something that will be
managed by you, it will be the Windows admins (although you would be
responsible for making the User Archive schedule available to them).

You _might_ be able to come up with some insane scheme like using a
scheduled backup that touches the client system, causes the clientside
script to run, and then uses bpend_notify once it's done to do all the
other heavy lifting.  That still won't get you away from Windows
scripting.

Basically, what you are trying to do is something that NBU is not meant
to do.  The expected behavior for Archives is that they are managed by
the client admin, not the backup admin.  Archive jobs get a filelist
from the user, run the archive, and delete the files when they are done,
so the user would already know what files were archived because they
would have supplied the list to the command in the first place.

You would, at least, be able to query the backup info from the master
the following day and report what was backed up, but that's about it.

--
David Rock
da...@graniteweb.com




Athena(r), Created for the Cause(tm)
Making a Difference in the Fight Against Breast Cancer

-
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential 
information and is for the sole use of the intended recipient(s). If you are 
not the intended recipient, any disclosure, copying, distribution, or use of 
the contents of this information is prohibited and may be unlawful. If you have 
received this electronic transmission in error, please reply immediately to the 
sender that you have received the message in error, and delete it. Thank you.
--

___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

Re: [Veritas-bu] Archive list

[David Rock]

* Dennis Peacock  [2012-03-08 07:14]:
> Patrick,
> I'm not a windows guy, so I'm trying to see what I can do without
> having to find someone that can write a "windows script" to do this.
> 
> What I'd prefer to be done is for me to setup a backup policy in NBU
> that is an Archive type policy. Point that policy to
> D:\Archive_Data\LZ and schedule the backup to run every night at 22:00
> hours. When the backup finishes successfully, it would remove the
> files and then run the bpflist or whatever to generate a list of what
> was backed up and email that to the customer.
> 
> That's what I'd prefer to do, but I'm lacking in knowledge in this
> windows stuff as I've been living in the Unix/Linux world for over 30
> years.  [Wink]

Well, 

First of all, there is no such thing as an "Archive" Policy type.  The
closest thing you would have is a Policy type of "MS-Windows" with a
schedule type of "User Archive".  Archives are designed to be User
initiated, not Scheduled from the Master Server.

As for notifications, especially because of the nature of Archives, that
is something that is going to pretty much _have_ to be a Windows script
on the client system.  You would have the logic in the clientside script
to manage all of that.  There are a couple likely issues:

1. The Windows server doesn't have an MTA to send email with (typically
there is no sendmail or equivalent on Windows, so that would need to be
added)

2. It's pretty much guaranteed that this won't be something that will be
managed by you, it will be the Windows admins (although you would be
responsible for making the User Archive schedule available to them).

You _might_ be able to come up with some insane scheme like using a
scheduled backup that touches the client system, causes the clientside
script to run, and then uses bpend_notify once it's done to do all the
other heavy lifting.  That still won't get you away from Windows
scripting.

Basically, what you are trying to do is something that NBU is not meant
to do.  The expected behavior for Archives is that they are managed by
the client admin, not the backup admin.  Archive jobs get a filelist
from the user, run the archive, and delete the files when they are done,
so the user would already know what files were archived because they
would have supplied the list to the command in the first place.

You would, at least, be able to query the backup info from the master
the following day and report what was backed up, but that's about it.

-- 
David Rock
da...@graniteweb.com


pgpoF2yhXmc83.pgp
Description: PGP signature
___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

Re: [Veritas-bu] Archive list

[Lightner, Jeff]

bpflist can be used to show what files are in a given backup.

You can get a list of backup IDs by running:
bpimmedia -L -policy   -d MM/DD/ hh:mm:ss -e MM/DD/ hh:mm:ss 
|grep _

The backup ID typically has _ followed by epoch date number – 
sometimes it isn’t hostname but virtual as in the case of Windows SQL backups.
The –d = start of range and the –e = end of range – this doesn’t have to be 
exact for above.

Once you know the image you can run:
bpflist -backupid atubks01_1301811357 -d MM/DD/ hh:mm:ss -e MM/DD/ 
hh:mm:ss -rl 999

NOTE:  If client/media server is not the master then add "-client ".
I don’t know why you have to specify date range with bpflist if you’re 
specifying the backup ID but you do.   It needs to be close to the original 
range of the backup.







From: veritas-bu-boun...@mailman.eng.auburn.edu 
[mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of Wilcox, Donald 
A (GE Global Research)
Sent: Monday, February 27, 2012 2:28 PM
To: VERITAS-BU@MAILMAN.ENG.AUBURN.EDU
Subject: [Veritas-bu] Archive list

Is there a way to get a file listing of all files backed up for an archive that 
I can mail to the customer so they can use it down the road to reference files 
needed?  I think the –L option only gives errors and warniings.





Athena®, Created for the Cause™

Making a Difference in the Fight Against Breast Cancer



How can I show my support for bottled water?

Take a minute to sign a petition in support of bottled water. Your signature 
COUNTS! Available through the 
bottledwatermatters.org/luv-bottled-water
 website, the goal is 50,000 signatures. Please include your personal email 
address when submitting. Once completed, share the website with your family and 
friends, to ensure as many bottled water supporters as possible sign the 
petition



-
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential 
information and is for the sole use of the intended recipient(s). If you are 
not the intended recipient, any disclosure, copying, distribution, or use of 
the contents of this information is prohibited and may be unlawful. If you have 
received this electronic transmission in error, please reply immediately to the 
sender that you have received the message in error, and delete it. Thank you.
--


___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu